Code Generation on HumanEval 0-shot
68.9Pass@1VAR
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VARBase Model=Qwen2.5-7B2025.02 | 68.9 | — | — | — | — | — | |
| BaseBase Model=Qwen2.5-7B2025.02 | 64.63 | — | — | — | — | — | |
| ALoLBase Model=Qwen2.5-7B2025.02 | 54.88 | — | — | — | — | — | |
| No watermarkshot count=0-shot2026.04 | 54.88 | — | — | — | — | — | |
| VOWshot count=0-shot, delta=2.02026.04 | 53.66 | — | — | — | — | — | |
| VOWshot count=0-shot, delta=2.52026.04 | 50 | — | — | — | — | — | |
| UPVshot count=0-shot2026.04 | 50 | — | — | — | — | — | |
| SelfHashshot count=0-shot2026.04 | 46.95 | — | — | — | — | — | |
| LeftHashshot count=0-shot2026.04 | 45.73 | — | — | — | — | — | |
| DPOBase Model=Qwen2.5-7B2025.02 | 33.54 | — | — | — | — | — | |
| VARBase Model=Llama2-7B2025.02 | 14.02 | — | — | — | — | — | |
| ALoLBase Model=Llama2-7B2025.02 | 12.8 | — | — | — | — | — | |
| DPOBase Model=Llama2-7B2025.02 | 7.32 | — | — | — | — | — | |
| RDFshot count=0-shot2026.04 | 3.66 | — | — | — | — | — | |
| BaseBase Model=Llama2-7B2025.02 | 3.05 | — | — | — | — | — | |
| PDWshot count=0-shot2026.04 | 0 | — | — | — | — | — | |
| AR SFTModel Scale=7B, One-token mode (τ=1.0)=true2026.04 | — | — | 78.7 | — | — | — | |
| AR SFT (10 ep)Model Scale=0.5B, Training Epochs=10, One-token mode (τ=1.0)=true2026.04 | — | — | 32.3 | — | — | — | |
| AR SFT (5 ep)Model Scale=0.5B, Training Epochs=5, One-token mode (τ=1.0)=true2026.04 | — | — | 35.4 | — | — | — | |
| Base (no fine-tune)Model Scale=0.5B, One-token mode (τ=1.0)=true2026.04 | — | — | 26.2 | — | — | — | |
| Base (no fine-tune)Model Scale=7B, One-token mode (τ=1.0)=true2026.04 | — | — | 82.9 | — | — | — | |
| Block DiffusionModel Scale=0.5B, Block Size (B)=4, One-token mode (τ=1.0)=true2026.04 | — | — | 31.7 | — | — | — | |
| DS-MTPModel Size=1.8B, Shots=0-shot2025.08 | — | — | — | 20.11 | 22.28 | 23.78 | |
| DS-MTPModel Size=7B, Shots=0-shot2025.08 | — | — | — | 24.58 | 27.01 | 30.18 | |
| Fast-dLLMModel=Dream, Decoding Algo=Fast-dLLM2025.12 | — | 180 | 55.5 | — | — | — | |
| LoPAModel=Dream, Decoding Algo=LoPA2025.12 | — | 290 | 53 | — | — | — | |
| LoPAModel=D2F-Dream, Decoding Algo=LoPA2025.12 | — | 630 | 56.1 | — | — | — | |
| MARS-0.5BModel Scale=0.5B, Block Size (B)=4, SFT Loss Configuration=True, One-token mode (τ=1.0)=true2026.04 | — | — | 40.2 | — | — | — | |
| MARS-0.5B w/o SFT lossModel Scale=0.5B, Block Size (B)=4, SFT Loss Configuration=False, One-token mode (τ=1.0)=true2026.04 | — | — | 33.5 | — | — | — | |
| MARS-7BModel Scale=7B, Block Size (B)=4, One-token mode (τ=1.0)=true2026.04 | — | — | 81.7 | — | — | — | |
| MTPModel Size=1.8B, Shots=0-shot2025.08 | — | — | — | 30.7 | 32.82 | 34.76 | |
| MTPModel Size=7B, Shots=0-shot2025.08 | — | — | — | 35.21 | 38.45 | 42.5 | |
| NTPModel Size=1.8B, Shots=0-shot2025.08 | — | — | — | 30.41 | 31.98 | 33.53 | |
| NTPModel Size=7B, Shots=0-shot2025.08 | — | — | — | 36.21 | 39.1 | 42.68 | |
| TOPModel Size=1.8B, Shots=0-shot2025.08 | — | — | — | 32.35 | 34.96 | 38.41 | |
| TOPModel Size=7B, Shots=0-shot2025.08 | — | — | — | 34.48 | 37.74 | 41.77 | |
| VanillaModel=Dream, Decoding Algo=Vanilla2025.12 | — | 100 | 55.5 | — | — | — | |
| VanillaModel=D2F-Dream, Decoding Algo=Vanilla2025.12 | — | 250 | 56.1 | — | — | — |