# Denoising with a Joint-Embedding Predictive Architecture

Source: [Denoising with a Joint-Embedding Predictive Architecture](<https://www.sota2.com/research/paper/68fa76d51e209131cea6f64e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:03:16.343Z

Authors: Dengsheng Chen, Jie Hu, Xiaoming Wei, Enhua Wu

Published: 2024-10-02T05:57:10.000Z

## Abstract

Joint-embedding predictive architectures (JEPAs) have shown substantial promise in self-supervised representation learning, yet their application in generative modeling remains underexplored. Conversely, diffusion models have demonstrated significant efficacy in modeling arbitrary probability distributions. In this paper, we introduce Denoising with a Joint-Embedding Predictive Architecture (D-JEPA), pioneering the integration of JEPA within generative modeling. By recognizing JEPA as a form of masked image modeling, we reinterpret it as a generalized next-token prediction strategy, facilitating data generation in an auto-regressive manner. Furthermore, we incorporate diffusion loss to model the per-token probability distribution, enabling data generation in a continuous space. We also adapt flow matching loss as an alternative to diffusion loss, thereby enhancing the flexibility of D-JEPA. Empirically, with increased GFLOPs, D-JEPA consistently achieves lower FID scores with fewer training epochs, indicating its good scalability. Our base, large, and huge models outperform all previous generative models across all scales on ImageNet conditional generation benchmarks. Beyond image generation, D-JEPA is well-suited for other continuous data modeling, including video and audio.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Class-conditional Image Generation on ImageNet 256x256](<https://www.sota2.com/research/sota/class-conditional-image-generation-on-imagenet-256x256.md>): Class-conditional Image Generation; ImageNet 256x256; Inception Score (IS): 303.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2410.03755>)

[PDF](<https://arxiv.org/pdf/2410.03755.pdf>)
