arXiv study steers Qwen3-30B-A3B output by biasing mixture-of-experts routing
Researchers report that adding fixed biases to router logits can push tokens along chosen expert paths, changing a model's output format and shifting concept probabilities.
In plain words
Mixture-of-experts models send each token to different specialist subnetworks, called experts, across layers. The authors forced tokens toward experts that a chosen concept or output format usually uses, by biasing the router's scores. This changed the model's output format and shifted concept-related probabilities, though concept shifts rarely changed the final text.
What was announced
In an arXiv paper, the authors study mixture-of-experts (MoE) language models, where a token is routed to a sequence of experts across layers, and ask what happens when tokens are forced onto chosen expert paths. They build a target path from the experts that a concept's or role's tokens most often use, then add a fixed bias to each target expert's router logit during generation to steer tokens toward that path. They distinguish format targets, which concern the form of the output, from concept targets, which concern its topic.
In Qwen3-30B-A3B, the authors report that steering a format path changes the text the model writes, while steering a concept path shifts next-token probabilities toward the concept even when the path is built in another language, yet has little effect on the generated text. Steering only part of the generation suggests a format, once started, reinforces itself. Religion and money steer most strongly, the effect concentrates in the middle layers, and the authors probe whether embedding distinctiveness and word frequency explain variation across concepts.
What it does not show
- The evidence is an abstract only, so methods, full results and caveats in the paper are not covered here.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.