BugGen 2.0 shifts LLM role in RTL bug dataset generation
The arXiv paper reports that deterministic algorithms now handle target selection and validation, while the language model only rewrites code, cutting token use sharply in same-model comparisons.
In plain words
Chip verification teams need collections of known bugs to test whether debugging tools work. BugGen 2.0 uses ordinary software rules to pick where to inject a bug and validates the result, while a large language model only writes the faulty code itself. The authors say this split lowers cost and improves consistency.
What was announced
BugGen 1.0 relied on the language model both to choose mutation sites and to generate the edits, which the authors say raised inference costs and sometimes produced invalid choices. Version 2.0 assigns structural analysis, target selection, and validation to deterministic algorithms, leaving the model to perform localized semantic rewriting. Across four OpenTitan IP blocks and FabScalar, tested with three LLMs, the system met every requested quota of compilable bug scenarios that cause at least two test failures, with 14-16 of 16 configured mutation types represented.
In completed same-model comparisons, the authors report token-use reductions of 6.8x-15.4x versus BugGen 1.0 and 1.5x-7.9x versus Firefly. The system also expands datasets without simulating every new bug: after finding accepted scenarios with observable failures, it generates alternative mutations at the same locations, simulates only a subset, and estimates acceptance for the rest, skipping 63.6% of AES and 33.9% of FabScalar variant simulations. Full simulation found 77.7% of AES and 95.2% of FabScalar variants accepted, with a 1.09% estimation error on the directly comparable FabScalar set.
What it does not show
- Evidence is limited to the paper's abstract; full methods, benchmarks, and caveats are not available for verification.
- The token-use comparisons cover only completed same-model comparisons against BugGen 1.0 and Firefly; no independent replication is reported.
- The acceptance-rate estimation error is reported only for the directly comparable FabScalar set, not for AES.
For security and IT teams
- Scope
- The reported results cover four OpenTitan IP blocks plus FabScalar, tested with three LLMs, and do not extend to other designs in the abstract's evidence.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.