← News

ResearchSources: arXiv · 16 Nov 2026

Researchers report first benign overfitting result for trained finite-width ReLU networks

A new arXiv paper analyzes two-layer ReLU networks trained by gradient flow on least-squares regression, claiming the first benign overfitting result for finite-width ReLU networks with arbitrary regression functions.

In plain words

Benign overfitting is when a model fits training data very closely, even its noise, yet still predicts well on new data. The authors say they proved this can happen for a two-layer ReLU neural network trained by gradient flow, without assuming anything about the underlying function except that it and the noise are bounded.

What was announced

The paper studies least-squares regression with a fully connected two-layer network using ReLU activation, trained by gradient flow. The authors report a generalization result that requires no assumptions on the underlying regression function or the noise beyond boundedness. They work in the neural tangent kernel regime and decompose excess risk into estimation and approximation errors, treating gradient flow as an implicit regularizer; they describe this decomposition as a novel perspective that helps avoid uniform convergence traps.

Under the same setting, the authors also establish that the trained network overfits to the data. Combining the two findings, they state this yields the first result on benign overfitting for finite-width ReLU networks with arbitrary regression functions. The claims come from the paper's abstract alone, so the proofs and their assumptions cannot be checked from this excerpt.

What it does not show

  • Evidence is limited to the paper's abstract; the full proofs, assumptions and scope of the results are not available for verification.
  • The 'first result' claim is the authors' own characterization and has not been independently confirmed.

Sources

Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 17 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.