← News

Agents & MCPSources: arXiv · 19 Nov 2026

Coding agents train optical neural networks from camera measurements alone

Researchers report an 'optics as code' method in which a large language model agent tunes physical optical hardware directly, without calibrated digital models or gradient-based training.

In plain words

Optical neural networks are computing devices that use light instead of electronics, and each physical device behaves slightly differently from its design, so it must be trained while connected to the real hardware. The researchers had a coding agent — an AI model that writes and runs programs — train such a device using only camera images of its own experiments. The agent learned to describe the optics in terms it could interpret and adjust, performing well without the usual per-device calibration step.

What was announced

The arXiv paper's abstract states that in situ training of optical neural networks has so far depended either on an expert-built calibrated digital model of each device or on gradient-free trial and error. The new method, called optics as code, instead places a coding agent in a training loop: the agent requests experiments, receives camera images of the results, and must state a hypothesis with each request. According to the authors, the agent developed a physical understanding of the optics and reparametrized the network using interpretable optical elements tuned directly on the hardware.

On the reported results, the approach outperformed gradient-free methods in classification and image encoding tasks and came close to backpropagation through a calibrated model while costing a fraction as much, the authors write. They add that the method needs neither per-device calibration nor gradients through the hardware, and that performance improves as the underlying language models become more capable. The authors present this as a route toward larger, scalable optical neural networks and as a general tool for calibrating optical experiments. These claims come from the paper's abstract and have not been independently verified.

What it does not show

  • The evidence is limited to the paper's abstract; no full-text details, benchmark numbers, or third-party replication are available.
  • Performance comparisons are the authors' own reported results and do not constitute independent confirmation.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.