ServiceExplorer graph lifts code localization for SWE agents under zero-leakage benchmark
A preprint reports a hierarchical dependency graph that consistently beat baseline and flat AST context across six models, using a benchmark protocol built to block temporal contamination and solution leakage.
In plain words
Software engineering agents need to find the right parts of a codebase to change. The paper's authors built ServiceExplorer, a two-level map of how code components depend on each other, and fed it to models as extra context. In their tests this helped models locate the correct code more often than working with no map or a flatter one.
What was announced
The arXiv preprint argues that standard benchmarks for repository-aware agents suffer from temporal contamination and prompt-level solution leakage, meaning models can benefit from knowledge that postdates the code state being tested. To prevent this, the authors lock repository snapshots and knowledge graphs at historical cutoff commits and evaluate 100 later pull requests in DragonFly (C++) and React (JavaScript/TypeScript), with four prompt granularities from Minimal to Guided.
Under this protocol they compared three context regimes: an unaugmented baseline, a flat Tree-sitter AST graph, and ServiceExplorer, which captures system and package containers plus architectural modules. Across 7,200 paired evaluations covering six foundation models from the MAI-Code Flash, Claude, and GPT-Sol families, the authors report ServiceExplorer outperformed both alternatives for every model, repository, and prompt granularity, with relative F1 lifts of 15.6% to 86.2% and a reported p-value below 10^-4. It also tripled exact full-PR localization on DragonFly. A further claim is that an older or efficiency-tier model augmented with ServiceExplorer beat the unaugmented F1 of the next-generation model in every family evaluated.
What it does not show
- All figures come from the paper's own abstract; no independent replication is reported.
- The abstract covers only two repositories (DragonFly and React) and six models from three named families, so results may not generalize to other codebases or models.
- The abstract does not describe how ServiceExplorer is built, distributed, or integrated, so practical availability is unknown.
- Statistical significance is asserted via a p-value threshold, but effect sizes depend on the paper's own evaluation setup.
For security and IT teams
- Scope
- Findings apply to the two tested repositories, DragonFly and React, and the six evaluated models, under a zero-leakage benchmark protocol.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.