Study finds protocol filtering improves how LLMs read network traffic
A new arXiv paper proposes a question-answering benchmark for testing large language models on packet captures, and reports that filtering traffic by protocol cuts prompt size while raising answer accuracy.
In plain words
Network engineers capture raw traffic records, called packet traces, to diagnose problems, but these files are hard to interpret. The authors tested whether large language models can answer questions about such traces reliably. They found that pre-filtering the data to a single network protocol made the models' answers more accurate and the prompts smaller.
What was announced
Researchers behind the arXiv paper built a reproducible evaluation that quizzes models on network traces and checks the answers against a human-generated ground truth. They used benign, real-world traffic from the CICIDS2017 dataset and compared how several models handled raw textual traffic dumps versus protocol-filtered representations. Scoring was automated with an LLM-as-a-judge pipeline, which the authors validated against human evaluations.
The reported result is that protocol filtering significantly reduced prompt overhead while improving accuracy, because fewer incorrect responses were produced. The authors present this as evidence that protocol-aware filtering is a practical optimization for LLM-assisted network management. The abstract does not name the models tested, give accuracy figures, or say whether the findings extend to malicious or mixed traffic, since only benign CICIDS2017 data was used.
What it does not show
- The source is the paper's abstract only; it names no specific models and reports no numerical accuracy results.
- Evaluation used benign real-world traffic from CICIDS2017, so performance on malicious or mixed traffic is not established in the supplied evidence.
- The accuracy claim comes from the authors' own benchmark and LLM-as-a-judge scoring, validated by the authors against human evaluation; no independent confirmation is available.
For security and IT teams
- Scope
- The findings apply to question answering over benign CICIDS2017 packet captures, not to a deployed traffic-analysis product.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.