← News

ResearchSources: arXiv · 19 Nov 2026

PhoneValue-Bench grades AI trade-in inspection with dual ground truth

A new benchmark from arXiv targets how well AI models can price used phones from seller photos and diagnostics before the device ships.

In plain words

When people sell a used phone to a trade-in service, the company must estimate a payout from photos and diagnostic screenshots before receiving the device. The authors built a benchmark that scores AI systems on how completely they fill in such inspection reports. Their reference pipeline outperformed a standard vision-language model, though both struggled on the operational side.

What was announced

The benchmark, described in an arXiv paper, has two parts: a large training set of 21,550 trade-in orders with 278,748 images, and a small audited leaderboard cut of 335 quality-checked orders. Systems are scored by CardScore, which the authors say measures how completely a report is filled in, not whether the quoted price is accurate. Cosmetic and operational fill rates serve as diagnostic breakdowns, while grade and price predictions stay off the leaderboard.

The authors also present ValueAgent, a six-stage reference pipeline that standardizes photo views, detects regional defects, reads diagnostic screens, and combines the evidence under warehouse-style rules. On the same leaderboard cut and a frozen scorer, it reached 84.1 CardScore versus 69.9 for the strongest flat vision-language model, with the biggest difference in cosmetic fill. The authors note that operational fill stayed limited for both approaches under the two declared input contracts, and that all figures come from the paper's own evaluation rather than outside testing.

What it does not show

  • The evidence is the paper's abstract only; the full methodology, data collection and results have not been reviewed.
  • CardScore reflects report completeness, not payout accuracy, so the scores do not show whether quotes would be correct.
  • The reported comparison is the authors' own evaluation on their leaderboard cut; no independent confirmation is available.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.