← News

ResearchSources: arXiv · 20 Nov 2026

Shorter examples stretch prompt budgets further, arXiv study finds

A new arXiv paper argues that when token budgets are fixed, many brief demonstrations often beat a few long ones in in-context learning.

In plain words

In-context learning means adding worked examples to a prompt so a language model performs better without retraining. The paper asks how to spend a fixed prompt budget: fewer long examples or more short ones. Across two question-answering benchmarks and six models, shorter examples generally worked better.

What was announced

The authors frame prompt construction as a resource-allocation problem, since longer prompts raise inference cost and latency. Rather than asking how many examples to include, they test whether a budget is better spent on a few long demonstrations or many short ones. They sorted candidate demonstrations into length-based bins and built prompts from examples within the same length range under fixed budgets.

Experiments on the Beat the AI and Natural Questions benchmarks, using six instruction-tuned LLMs, showed shorter demonstrations were generally the most effective use of prompt space. The effect was strongest on Beat the AI; on Natural Questions the gains were smaller and saturated earlier in the long-context setting. An additional natural language inference experiment on ANLI R3 found one of the two shorter bins performed best for five of six models, though longer demonstrations stayed competitive. The authors conclude that demonstration granularity is an important factor in prompt design.

What it does not show

  • The evidence is the paper's abstract only; full methods, datasets details and peer review status are not visible.
  • Results are limited to the tested QA benchmarks, six instruction-tuned models, and one ANLI R3 experiment; no other tasks or model families are covered.
  • The findings are the authors' own reported results and have not been independently confirmed.

For security and IT teams

Scope
The reported gains apply to question-answering benchmarks and one ANLI R3 inference experiment, not to prompt design generally.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.