← News

Models & toolsSources: arXiv · 19 Nov 2026

ThinkSum pairs LLM queries with probabilistic inference to lift reasoning on BIG-bench tasks

The authors report a two-stage method that queries a model over sets of phrases and aggregates the results, improving on direct prompting across thirteen hard tasks.

In plain words

Large language models often struggle when a question requires combining many facts through several logical steps. ThinkSum breaks such a question into many smaller queries, then combines the answers using probability rules outside the model. The authors say this improves accuracy and makes predictions easier to interpret.

What was announced

In a paper on arXiv, the researchers describe ThinkSum as a two-stage inference paradigm. The first stage, Think, queries a large language model in parallel over a set of phrases drawn from the prompt or an auxiliary model call. The second stage, Sum, aggregates those query results through probabilistic reasoning to produce the final prediction, so the aggregation happens outside the model calls themselves.

Testing on the BIG-bench evaluation suite, the authors report improvements over the state of the art with GPT-family models on thirteen difficult tasks, often using far smaller model variants. They also compare ThinkSum with chain-of-thought prompting variants and state that, because inference occurs outside the LLM, the approach is less sensitive to prompt design, yields more interpretable predictions, and can combine with latent variable models. The findings come from the paper's abstract, which does not include the underlying benchmarks' details, error rates, or which specific GPT models were used.

What it does not show

  • Evidence is limited to the paper's abstract; no full results, task-level scores, model versions, or independent replication are available to verify the claimed improvements.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.