Researchers demo a small language model that runs fully in the browser tab
A fine-tuned 0.8B model converts typed orders into structured data on the visitor's own device, with no text sent to a server, and the authors invite the public to hunt for its silent mistakes.
In plain words
The authors built a web page where a small language model turns typed orders into structured records while running entirely on the visitor's computer. Nothing typed is sent to a server. The point is to let people test where the model produces tidy-looking output that does not match what was actually ordered.
What was announced
The arXiv abstract describes two failure modes for a model that structures typed orders. One produces visibly broken output that any system rejects; the other, which the authors' companion work calls a silent error, is a well-formed record that nobody actually placed, and the authors flag it as the dangerous kind. Their demo runs a fine-tuned 0.8 billion parameter model inside the browser tab via llama.cpp compiled to WebAssembly, so no order text leaves the device. The page parses input as the visitor types, scores each ticket with a TypeScript port of the PIZZA benchmark's matcher, and marks anything off the menu with a question mark.
On the reported numbers, the browser-deployed model matched the server evaluation's output on 99.0% of the 1,357 PIZZA test orders and reached the same exact match score of 0.882. The authors also report latency measured on three consumer laptops. They frame the demo as a way for visitors to search for silent errors or attempt to trigger one, and note this deployment lets the model face the open web.
What it does not show
- The evidence is an abstract only; the full paper's methods, error analysis and hardware details are not covered here.
- Latency figures are reported for three consumer laptops, so results may not generalise to other devices.
- The 99.0% agreement and 0.882 exact match are the authors' own reported results, not independently confirmed.
For security and IT teams
- Scope
- The demo processes order text entirely on the visitor's device, and the authors state no order text leaves it.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.