# RiT: Vanilla Diffusion Transformers Suffice in Representation Space

Source: [RiT: Vanilla Diffusion Transformers Suffice in Representation Space](<https://www.sota2.com/research/paper/6a0fe327f984739ed6739e89>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-24T06:00:01.510Z

Authors: Le Zhang, Ning Mang, Aishwarya Agrawal

Published: 2026-05-21T04:21:43.000Z

## Abstract

Flow matching with $x$-prediction -- regressing the clean data point rather than the ambient velocity -- is known to exploit low-dimensional manifold structure effectively in pixel space \\cite{li2025back}. We ask whether a pretrained representation space, while containing a low-dimensional data manifold of comparable intrinsic dimensionality, offers a distribution more favorable for flow-matching learning. Comparing pixel, SD-VAE, and DINOv2 features along four geometric axes, we find that pixel and DINOv2 share nearly identical intrinsic dimensionalities (both $\\hat{d}\\!\\approx\\!33$) yet DINOv2 exhibits $7.3\\times$ higher effective rank, $35\\times$ better covariance conditioning, $11.5\\times$ lower excess kurtosis, and $1.7\\times$ lower on-manifold interpolation error; SD-VAE latents are consistently intermediate, indicating that the advantage stems from representation-learning objectives rather than mere compression. These statistical properties render the flow-matching regression well-conditioned and remove the need for the specialized prediction heads or Riemannian transport used by prior DINOv2 diffusion methods. We propose the \\emph{Representation Image Transformer} (RiT): a vanilla Diffusion Transformer trained by $x$-prediction on frozen DINOv2 features, augmented only by a dimension-aware noise schedule and joint \\texttt{\[CLS\]}-patch modeling. On ImageNet $256{\\times}256$, RiT attains FID 1.45 without guidance and 1.14 with classifier-free guidance, outperforming DiT$^\\text{DH}$-XL with $19\\%$ fewer parameters (676M vs.\\ 839M). The resulting ODE is efficiently solvable at coarse discretizations: with classifier-free guidance, $5$ Heun steps already reach FID 2.0 and $10$ steps reach 1.25, without distillation or consistency training. Code at https://github.com/lezhang7/RiT.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Class-conditional Image Generation on ImageNet 256x256](<https://www.sota2.com/research/sota/class-conditional-image-generation-on-imagenet-256x256.md>): Class-conditional Image Generation; ImageNet 256x256; Inception Score (IS): not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.21981>)

[PDF](<https://arxiv.org/pdf/2605.21981.pdf>)

[Implementation](<https://github.com/lezhang7/RiT>)
