HiT paper proposes hierarchical token grouping for pixel-space image generation
A new arXiv paper introduces Hierarchical Diffusion Transformers, which group image tokens at multiple scales and report stronger pixel-space generation with less training compute.
In plain words
The authors of this paper propose a design change for image-generating AI models. Their method, HiT, merges similar image pieces into single summary tokens as data passes through the network, then restores them afterward. They report this improves quality and efficiency without a separately pretrained visual encoder.
What was announced
Pixel-space diffusion models learn to represent and generate images in one end-to-end training stage, avoiding a separately pretrained visual encoder, but the authors state that plain pixel-space Transformers still trail leading two-stage latent diffusion models in quality and training efficiency. They attribute part of this gap to keeping one token per patch throughout the backbone, which they argue does not encourage a visual hierarchy summarizing content at coarser levels. HiT addresses this by progressively grouping related tokens, with a single token representing each group in the main Transformer bottleneck; ungrouping restores features to their original image positions. A compression budget regulates the average compression ratio, and group memberships and counts vary across images and diffusion timesteps during training and generation.
The authors report that the model develops emergent semantic structure, grouping coherent objects and object parts without segmentation labels, and in text-to-image generation it groups caption words with matching image patches without region annotations. On measured results, HiT achieves 1.48 FID on ImageNet 256x256 and reaches generation quality comparable to JiT using roughly 40% of its cumulative training FLOPs. For text-to-image generation, the paper says HiT exceeds UniLiP's average score across five benchmarks with about 6x fewer denoiser parameters and 3x fewer forward FLOPs. The authors conclude that learning a visual hierarchy inside the generator can strengthen image generation without external visual pretraining.
What it does not show
- Evidence is limited to the paper's abstract; the full methodology, experimental setup and peer review status are not covered, and the reported benchmarks and comparisons are the authors' own, not independently confirmed.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.