← News

ResearchSources: arXiv · 19 Nov 2026

Study finds Speech LLMs handle noisy spoken commands better than ASR-LLM cascades

An arXiv paper compares two ways of turning speech into structured outputs and reports that end-to-end speech models hold up better under noise, though not on every measure.

In plain words

Some AI systems convert spoken instructions into structured data such as intent labels or function calls. The paper compares two designs: models that process speech directly, and pipelines that first transcribe speech with automatic speech recognition (ASR) before a language model reads the text. The authors report that the direct speech models performed better overall, especially when the audio was noisy.

What was announced

The paper, posted on arXiv, examines spoken semantic parsing: generating structured outputs such as semantic frames and function calls from speech rather than text. The authors compare Speech LLMs, which take audio directly, against ASR-LLM cascades, which transcribe audio first and then parse the transcript. Both approaches were adapted using text-only task supervision, meaning no speech-specific task labels, and tested with 3B and 7B parameter models across three datasets. Evaluation covered clean speech and babble noise at five signal-to-noise ratio levels.

According to the results, Speech LLMs generally performed better on command understanding, intent prediction, and function calling, with the advantage most pronounced under severe noise. Gains on fine-grained entity prediction were less consistent, so the direct-speech advantage does not extend uniformly to every subtask. A separate evaluation also found that Speech LLMs had lower average inference time per sample than the cascades at both model scales. The findings come from the authors' own experiments and have not been independently confirmed.

What it does not show

  • The evidence is the paper's abstract only, so full details on datasets, model configurations, and error analysis are not available in the supplied passages.
  • The comparison covers babble noise at five SNR levels and three datasets with 3B and 7B models; results for other noise types, model sizes, or domains are not established by the supplied text.
  • The reported performance and speed advantages come from the authors' own evaluation and are not independently confirmed.

For security and IT teams

Scope
The comparison is limited to spoken semantic parsing tasks (semantic frames, intent prediction, function calling) under clean speech and babble noise, using 3B and 7B models across three datasets.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.