← News

ResearchSources: arXiv · 18 Nov 2026

Nonsense strings can outrank real passages in dense retrievers, study finds

A new arXiv paper, NONSENSE, shows that gibberish strings optimized for similarity often beat genuinely relevant passages in popular retrieval models.

In plain words

Dense retrieval is the technique behind many search and question-answering systems: a model gives text a numeric score, and items with high scores are treated as semantically relevant. The NONSENSE authors built meaningless strings that score highly against a query without copying its words. In their tests, these strings often ranked above the best genuinely relevant passage, suggesting the scores do not always track meaning.

What was announced

The paper describes NONSENSE as a black-box probe that makes small, sparse edits to text, producing strings that look meaningless to people. The method is not allowed to copy words from the query; it only maximizes the similarity score a retriever assigns between the two. Across five BEIR-style datasets and three widely used single-vector retrievers, these strings frequently earned higher scores than the best judged query-relevant passage for that retriever, and they often reached the top rank in the whole corpus.

The authors read this as evidence of a semanticity gap: a model's nearest-neighbor ranking can be pulled apart from whether a text actually carries passage-like meaning. They argue that evaluation should check not just whether relevant documents come back, but how faithfully similarity scores mirror semantic evidence. Because the work is presented as an abstract, the full paper's experiments and any rebuttals are not covered here.

What it does not show

  • Only the paper's abstract is available, so the full experimental detail, model versions and dataset specifics cannot be verified from this excerpt.
  • The findings cover five BEIR-style datasets and three single-vector retrievers named in the abstract; performance of other retriever types is not established.
  • No independent replication of the results is reported in the supplied material.

For security and IT teams

Scope
The reported results apply to the three single-vector retrievers and five BEIR-style datasets tested in the paper, not to retrieval systems generally.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.