Minitap's mobile-use multi-agent system reports 100% pass@3 on AndroidWorld
The open-source Android automation system splits each task across five LLM agents plus deterministic nodes, and its authors report near-perfect success on held-out AndroidWorld task instances.
In plain words
Minitap's mobile-use is an open-source system that lets AI agents control Android phones by breaking each task into five cooperating language-model roles, such as planning and execution, alongside fixed non-AI steps. Its authors report it completed every tested AndroidWorld task on fresh instances, though results come from the developers' own evaluation.
What was announced
Minitap's mobile-use assigns each Android task to five LLM agents covering planning, subgoal tracking, state capture, decision and execution, combined with deterministic nodes. Supporting mechanisms read typed text back from the device, verify the goal against on-device evidence and detect repeated failures. On 116 AndroidWorld tasks run on emulators in Minitap's cloud infrastructure, real Pixel 7 phones from its device farm and the stock emulator, the authors report a pass@3 of 100.0% (95% CI 96.8-100.0) on fresh, held-out task instances from three pre-registered seeds never used during development, with pass^3 at 94.0% and mean success at 98.0%.
An ablation of the initial system attributes much to decomposition: removing it changes success by -26.1 points, while the verification mechanisms add +11.7 points over a single agent; the decision agent is described as the only role needing a frontier model. On MobileMiniWoB++, a second benchmark held out from development, the final system improved on the initial one by 7.1 points (McNemar p = 0.035; CI -0.5 to +14.7). The code is available on GitHub.
What it does not show
- All results are self-reported by Minitap from its own paper and infrastructure; no independent replication is cited.
- The MobileMiniWoB++ improvement's confidence interval spans -0.5 to +14.7, so the effect is not firmly established.
- The source is an abstract, so full methodological detail is not available.
For security and IT teams
- Scope
- Evaluation covered AndroidWorld's 116 tasks on Minitap's cloud emulators, real Pixel 7 phones and the stock emulator, not arbitrary production Android environments.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.