ECOOP 2026
Mon 29 June - Fri 3 July 2026 Brussels, Belgium
Mon 29 Jun 2026 11:45 - 12:07 at I.2.02 - Specification and Analysis Chair(s): Jens Nicolay

Large language models are increasingly used to make static analysis tools accessible through natural language, yet existing systems differ in how much they delegate to the LLM without treating the degree of delegation as an independent variable. We compare three architectures along a spectrum of LLM involvement for translating natural language to Joern’s query language CPGQL: direct query generation A1, generation of a schema-constrained JSON intermediate representation A2, and tool-augmented agentic generation A3. These are evaluated on a benchmark of 20 code analysis tasks across three complexity tiers, using four open-weight models in a 2X2 design (two model families X two scales), each with three repetitions. The structured intermediate representation A2 achieves the highest result match rates, outperforming direct generation by 15–25 percentage points on large models and surpassing the agentic approach despite the latter consuming 8X more tokens. The benefit of structured intermediates is most pronounced for large models; for small models, schema compliance becomes the bottleneck. These findings suggest that in formally structured domains, constraining the LLM’s output to a well-typed intermediate representation and delegating query construction to deterministic code yields better results than either unconstrained generation or iterative tool use.

Mon 29 Jun

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

11:00 - 12:30
Specification and AnalysisGPCE at I.2.02
Chair(s): Jens Nicolay Vrije Universiteit Brussel
11:00
22m
Talk
Comparing Solver Representations for Analyzing Cardinality-Based Feature Models
GPCE
Fabian Eger Karlsruhe Institute of Technology, Lukas Güthing Karlsruhe Institute of Technology, Kevin Feichtinger Karlsruhe Institute of Technology (KIT), Ina Schaefer KIT
11:22
22m
Talk
ATLAS: From Access conTrol Language to ACSL Specifications
GPCE
Julien Signoles Université Paris-Saclay, CEA, List, Khaoula Boukir Ibn Tofail University, Amine Nasri Université Paris-Saclay, CEA, List & Ibn Tofail University
11:45
22m
Talk
Less Is More: Measuring How LLM Involvement Affects Chatbot Accuracy in Static Analysis
GPCE
12:07
22m
Talk
TurtleTalk: A DSL for Constraint-Based Turtle Graphics in Programmatic CAD
GPCE
Jef Jacobs Vrije Universiteit Brussel, Wolfgang De Meuter Vrije Universiteit Brussel, Jens Nicolay Vrije Universiteit Brussel