Formal Context Protocol checks AI-generated artifacts against the original request
A research paper proposes FCP, a protocol that verifies whether AI-generated hardware and code actually meet the requirements they were written for, rather than relying on the model's own tests.
In plain words
When a language model writes both a program and the tests for it, a passing test proves little, because the same misunderstanding can shape both. The Formal Context Protocol instead checks the artifact against a formal reading of the original request. Anything a machine cannot settle becomes a concrete question for the request's author.
What was announced
The paper, a cross-listed arXiv abstract, argues that self-testing is weak evidence: one misreading of a request can produce a faulty artifact and a check that accepts it. FCP splits verification into two parts. Conformance, whether the artifact satisfies a formal reading of a requirement, is decided by model checkers and SMT solvers. Fidelity, whether that reading matches the meaning of the prose, has no decision procedure but can be refuted by a concrete witness. Each of eight ways a proof can falsely certify a requirement becomes a mechanical check, and adapters place hardware, text, agents and code on the same engines.
In reported results, trusting AI-written assertions on four hardware designs certified 23% of violated requirements, while FCP certified none. Across ten code tasks, code from three language models passed its own tests yet violated the request every time; trusting the tests certified all 33 violations, while FCP certified three. The authors note that agreement among models, readings or tests did not imply correctness. The figures come only from the abstract, which does not describe the task selection, the models' identities or how the protocol performs outside these fourteen cases.
What it does not show
- Evidence is limited to the paper's abstract; experimental setup, model identities and peer review are not described, and the reported results are not independently confirmed.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.