← News

ResearchSources: arXiv · 18 Nov 2026

AlignedHPO tunes loss hyperparameters with gradient alignment, reports 12x faster hypergradients

An arXiv paper proposes a framework that treats differentiable loss hyperparameters as optimizable variables, using Jacobian-vector products and surrogate metrics to cut hypergradient cost.

In plain words

Deep learning loss functions often contain settings like weights, scales, and margins that teams usually tune by hand or with costly black-box search. AlignedHPO instead exploits the fact that these settings are differentiable, so gradients can guide them directly. The authors say this makes tuning faster while aiming at the evaluation metric that actually matters.

What was announced

The paper, posted to arXiv by the AlignedHPO authors, argues that modern loss functions carry multiple hyperparameters that are typically tuned manually or through expensive black-box optimization, even though they are differentiable. The framework computes hypergradients using Jacobian-vector products to reduce time and memory, and steers optimization with differentiable surrogate versions of task metrics. Normalized alignment and gradient smoothing are used to handle gradient magnitude ambiguity and stochastic instability, according to the abstract.

In experiments on generalized Focal Loss and ArcFace, the authors report effective adaptation across long-tailed classification and open-set face recognition. They claim performance competitive with or better than implicit differentiation, with hypergradient computation nearly 12 times faster. These are the authors' own reported results from the abstract; the excerpt does not include the full experimental detail, and no independent verification is available.

What it does not show

  • Evidence is limited to the paper's abstract; full methodology, experimental setup, and baselines are not covered by the supplied passages.
  • The speed and performance figures are the authors' self-reported claims and have not been independently confirmed.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.