# Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space

Source: [Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space](<https://www.sota2.com/research/paper/697ae987a2b28e761d87d052>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-07T05:00:04.108Z

Authors: Yangyi Shen, Tianjian Feng, Jiaqi Han, Wen Wang, Tianlang Chen, Chunhua Shen, Jure Leskovec, Stefano Ermon

Published: 2026-01-28T07:55:07.000Z

## Abstract

Diffusion Language Models (DLMs) offer order-agnostic generation that can explore many possible decoding trajectories. However, current decoding methods commit to a single trajectory, limiting exploration in trajectory space. We introduce Order-Token Search to explore this space through jointly searching over generation order and token values. Its core is a likelihood estimator that scores denoising actions, enabling stable pruning and efficient exploration of diverse trajectories. Across mathematical reasoning and coding benchmarks, Order-Token Search consistently outperforms baselines on GSM8K, MATH500, Countdown, and HumanEval (3.1%, 3.8%, 7.9%, and 6.8% absolute over backbone), matching or surpassing diffu-GRPO post-trained d1-LLaDA. Our work establishes joint search as a key component for advancing decoding in DLMs.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Mathematical Reasoning on MATH 500 (accuracy, average accuracy)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-500-accuracy-average-accuracy.md>): Mathematical Reasoning; MATH 500; Accuracy: 42.4.

- [Sudoku on Sudoku 4x4 (test)](<https://www.sota2.com/research/sota/sudoku-on-sudoku-4x4-test.md>): Sudoku; Sudoku 4x4 (test); Accuracy (Avg): 12.8.

- [Mathematical Reasoning on Countdown](<https://www.sota2.com/research/sota/mathematical-reasoning-on-countdown.md>): Mathematical Reasoning; Countdown; Accuracy: 34.4.

- [Code Generation on HumanEval](<https://www.sota2.com/research/sota/code-generation-on-humaneval.md>): Code Generation; HumanEval; Pass@1: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.20339>)

[PDF](<https://arxiv.org/pdf/2601.20339.pdf>)
