CoTTAGE: a reasoning judge that scores and explains text-to-audio output
Researchers describe an automatic evaluator for generated audio that reasons about errors before rating text relevance and audio quality, reporting top correlation with human judgments across six benchmarks.
In plain words
Text-to-audio systems turn written prompts into sound, and judging how good the results are usually needs people to listen and score them. CoTTAGE is an AI judge that first writes an explanation of what went wrong in a clip, then gives scores for prompt match and sound quality. The authors say this approach matches human ratings better than earlier metrics while also giving useful explanations.
What was announced
The team behind the arXiv paper argues that current evaluation metrics rarely combine two things at once: scores that track human opinions closely, and explanations grounded in the actual audio. That gap leaves developers leaning on manual listening tests, which the authors describe as a bottleneck that slows each round of model improvement. CoTTAGE targets the two common rating axes, text relevance and overall audio quality, and works by reasoning first: it produces a rationale tied to the audio to diagnose problems, then predicts a score.
Training relied on more than 10,000 newly gathered human ratings of overall quality. The model was first tuned with supervised fine-tuning, then improved with reinforcement learning. That second stage used a preference objective built on Thurstone's method, which turns sampled score distributions into pairwise preference probabilities and rewards agreement with rankings taken from human judgments. On six benchmarks covering both rating axes, the authors report state-of-the-art correlation with human scores plus explanations that people preferred. The claims come from the paper's abstract, which does not include the benchmark numbers, model details, code availability, or any independent replication.
What it does not show
- Evidence is limited to the paper's abstract; benchmark figures, model specifics and code availability are not provided.
- The reported results are the authors' own claims and have not been independently confirmed.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.