← News

Models & toolsSources: arXiv · 20 Nov 2026

arXiv tutorial explains what universal approximation theorems do and do not prove

A new arXiv paper reviews the mathematics behind neural network expressivity, stressing that approximation guarantees say nothing about training efficiency or generalization.

In plain words

Neural networks are widely used to model complicated patterns, and mathematicians have long proved they can in principle represent almost any function. The paper's key point is that such proofs do not tell engineers how big a network must be, whether training will succeed, or how well it will work on new data.

What was announced

The paper, posted as a cross-listed abstract on arXiv, recounts the classical universal approximation theorems of Cybenko, Funahashi, Hornik and Leshno, which justify why neural networks can model complex nonlinear phenomena. It notes that shallow networks can approximate any continuous function on compact sets, and that Leshno's theorem showed non-polynomial activations are necessary for universality, explaining why ReLU, logistic and hyperbolic tangent activations work while pure polynomials do not.

The author emphasizes the limits of these results: they give no efficiency, training or generalization guarantees, and do not indicate how many neurons a given accuracy requires or whether gradient descent will find suitable parameters. The review also cites Barron's quantitative error bounds for functions with bounded Fourier norm and Montúfar's result that expressivity grows exponentially with depth, measured in linear regions, making deep networks strictly more expressive than shallow ones with equal parameter counts.

What it does not show

  • The source is only the paper's abstract, so the full article's arguments, proofs and scope could not be checked.

Sources

Source-support review completed 21 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 21 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.