OPT* tasks train LLMs on optimization-style reasoning
Researchers introduce OPT*, a family of optimization-style tasks that scales search-space difficulty without new human labels, for training and evaluating step-by-step reasoning in large language models.
In plain words
Large language models are usually trained to reason on math and coding problems with checkable answers. The OPT* paper instead builds tasks where a model must find a high-value workable plan among many valid options. A complexity parameter widens the search space, so difficulty grows without extra human labeling.
What was announced
The authors frame OPT* as a scalable family of tasks for step-by-step optimization-like reasoning. Each task supplies a feasibility checker and an evaluator, and a complexity parameter expands the search space without requiring new human labels. The paper studies two regimes: solver-guided online policy optimization, where a solver acts as a value oracle for partial states and rank-based reward shaping reinforces better next steps, and search-based offline reinforcement learning for cases where such solvers are unavailable.
Theoretically, the paper links success in large search spaces to how much information a reasoner extracts per unit of search budget. Empirically, the authors ablate the ingredients that make search efficient on OPT* and report that training on the tasks improves step-by-step optimization-like reasoning. The claims come from the paper's abstract, so independent replication and full results are not covered here.
What it does not show
- Evidence is limited to the paper's abstract; the v2 replacement's full methods, benchmarks and results are not available for verification.
- The abstract states that training on OPT* improves optimization-like reasoning but gives no quantitative results, model sizes or baselines.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.