AgenticPay benchmark finds LLM agents struggle to negotiate in competitive markets
A new arXiv paper introduces AgenticPay, a benchmark testing whether multimodal AI agents can bargain and reach agreements in simulated commerce, and reports substantial weaknesses across the models tested.
In plain words
Researchers built a test set that asks AI agents to negotiate deals, such as prices and contract terms, in simulated marketplaces. The best model scored only 42.3 out of 100, and results dropped further as the markets became more crowded with competing agents.
What was announced
AgenticPay, described in a paper on arXiv, contains 160 multimodal tasks spanning four business scenarios: e-commerce, food delivery, ride-hailing, and apartment rental. The tasks cover eight market structures, ranging from one-on-one bargaining to many-to-many competition. Agents must interpret product images, infer opponents' hidden preferences, and negotiate binding terms such as price, lease duration, return policy, and delivery speed through multi-round dialogue. A utility-based framework scores agreement quality and joint welfare, the authors say.
The paper's experiments covered proprietary models (GPT-5.4, Claude Sonnet 4.6, and Gemini 3.1 Pro Preview) and open-weight models (Qwen3-VL-32B-Instruct and InternVL3-38B). The authors report substantial limitations in value creation and market reasoning: the strongest model, Gemini 3.1 Pro Preview, reached a GlobalScore of only 42.3 out of 100. As markets shifted from bilateral to multi-sided, the average GlobalScore fell by 5.7 points, with open-weight models declining by up to 11.5 points. Code and data are linked in the paper's abstract.
What it does not show
- The evidence is the paper's abstract only; the full methodology, experimental setup, and peer-review status are not available in the supplied passages.
- Scores and comparisons are the authors' own reported results from their benchmark; no independent evaluation is described.
For security and IT teams
- Scope
- AgenticPay is an evaluation benchmark for multimodal negotiation agents in simulated commerce scenarios, not a deployed trading system.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.