HackWorld benchmark finds computer-use agents exploit under 12% of web vulnerabilities
A new arXiv paper introduces HackWorld, a framework testing whether computer-use agents can find and exploit real web application flaws through visual interaction, and reports weak results.
In plain words
Computer-use agents are AI systems that operate software the way a person does, by looking at the screen and clicking. The HackWorld authors built 36 realistic web applications with planted security flaws and asked these agents to exploit them. Most agents failed, succeeding in fewer than 12% of cases.
What was announced
Researchers behind the HackWorld paper on arXiv describe it as the first framework for systematically evaluating computer-use agents on exploiting web application vulnerabilities through visual interaction. The benchmark contains 36 real-world applications built on 11 frameworks and 7 languages, with flaws including injection vulnerabilities, authentication bypasses, and unsafe input handling. A Capture-the-Flag setup tests whether agents can identify and exploit these weaknesses while navigating complex web interfaces, which the authors say demands visual understanding, dynamic content handling, and multi-step interactions that ordinary language model agents cannot perform.
Evaluations of state-of-the-art computer-use agents showed exploitation rates below 12% and low cybersecurity awareness. The authors report that agents often fail at multi-step attack planning and misuse security tools. They present these results as evidence of current limitations in web security contexts and as motivation for building more security-aware agents capable of vulnerability detection and exploitation. The paper also notes that traditional penetration testing is costly and expertise-intensive, which motivates automated approaches, though the reported success rates suggest such automation remains far from practical for offensive security work.
What it does not show
- The evidence is limited to the paper's abstract; detailed methodology, per-agent scores, and full results are not available in the supplied passages.
- The findings come from the paper's authors and have not been independently confirmed.
- The abstract does not specify which agents were tested or how the sub-12% figure was measured.
For security and IT teams
- Scope
- HackWorld covers exploitation of web application flaws by computer-use agents in a benchmark setting; it does not report on defensive tooling or production deployments.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.