← News

ResearchSources: arXiv · 20 Nov 2026

VR-SparseQuant pairs FP4 quantization with 2:4 sparsity on Nvidia Vera Rubin

An arXiv paper describes a method that keeps native FP4 execution on Vera Rubin GPUs while using elementwise 2:4 sparsity to recover accuracy lost in four-bit activation quantization.

In plain words

Running large language models on four-bit numbers saves memory and compute but can hurt output quality. The paper's method splits work into two sparse streams on Nvidia's Vera Rubin chips, spending the same value budget as dense quantization to fix the most error-prone activations. The authors report better accuracy and faster kernels than a BF16 baseline.

What was announced

The paper, posted to arXiv, argues that four-bit activation quantization can substantially degrade model quality even as GPU vendors add low-precision support such as FP8 on Hopper and FP4 Tensor Cores on Blackwell. Vera Rubin adds E2M1 FP4 with direct UE5M3 block scaling, which the authors call VRFP4. VR-SparseQuant overlaps two sparse streams so activations with larger errors can be refined within the same FP4 value budget that dense quantization would use. Scale ratios are calibrated offline to avoid runtime search, and a shared-weight GEMM accumulates both streams.

The reported results come from one model and one benchmark. On Nemotron-3-Nano-30B-A3B, the authors say VR-SparseQuant lowers WikiText-2 perplexity from 6.85 to 5.57 compared with dense RTN W4A4. Their native Rubin kernels for VRFP4, including activation preparation, reach up to 3.1x GEMM and 2.7x MoE speedup over BF16. The authors state this is, to their knowledge, the first study of adaptive activation quantization using Vera Rubin's elementwise 2:4 structured sparse FP4 instructions.

What it does not show

  • Evidence is limited to the paper's abstract; the full article was not reviewed, so method details and evaluation setup cannot be verified.
  • Results are reported for a single model (Nemotron-3-Nano-30B-A3B) and WikiText-2 perplexity; no other models or benchmarks are covered by the supplied evidence.
  • Speedup and accuracy figures are the authors' own claims and have not been independently confirmed.

For security and IT teams

Scope
The method targets Nvidia Vera Rubin hardware using its elementwise 2:4 structured sparse FP4 instructions and VRFP4 format, per the paper's abstract.

Sources

Source-support review completed 21 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 21 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.