Coding on HumanEval (test)
74.4Test AccuracyKL-SFT
Evaluation results
| Method | Links | |
|---|---|---|
| KL-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 74.4 | |
| BaseBackbone=Qwen2.5-3B-Instruct2026.05 | 73.8 | |
| STMBackbone=Qwen2.5-3B-Instruct2026.05 | 73.2 | |
| Low-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 72.6 | |
| Anchored LearningBackbone=Qwen2.5-3B-Instruct2026.05 | 70.7 | |
| DFTBackbone=Qwen2.5-3B-Instruct2026.05 | 70.1 | |
| Iter-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 70.1 | |
| SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 68.3 | |
| Self-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 67.1 | |
| Qwen3-4B-BaseNumber of shots=02026.06 | 65.24 | |
| FReDA-4BNumber of shots=0, Evaluation protocol=Best-of-N2026.06 | 63.41 | |
| FReDA-4BNumber of shots=0, Evaluation protocol=Self-refine2026.06 | 60.98 | |
| TiDAR-8BNumber of shots=0, Evaluation protocol=Trust Diff2026.06 | 57.93 | |
| Dream-7B-BaseNumber of shots=02026.06 | 57.9 | |
| BlockDiff-4BNumber of shots=02026.06 | 57.32 | |
| TiDAR-8BNumber of shots=0, Evaluation protocol=Trust AR2026.06 | 55.49 | |
| LLaDA-MoE-7B-A1B-BaseNumber of shots=02026.06 | 45.73 | |
| Qwen2.5-3B-BaseNumber of shots=02026.06 | 42.1 | |
| SPG w/ EUBOSequence Length=256, Evaluation Protocol=0-shot2025.10 | 41.5 | |
| UniGRPOSequence Length=256, Evaluation Protocol=0-shot2025.10 | 39.6 | |
| SPG w/ MixtureSequence Length=512, Evaluation Protocol=0-shot2025.10 | 39.6 | |
| WD1Sequence Length=256, Evaluation Protocol=0-shot2025.10 | 39 | |
| SPG w/ MixtureSequence Length=256, Evaluation Protocol=0-shot2025.10 | 39 | |
| WD1Sequence Length=512, Evaluation Protocol=0-shot2025.10 | 38.4 | |
| UniGRPOSequence Length=512, Evaluation Protocol=0-shot2025.10 | 38.4 | |
| D1Sequence Length=256, Evaluation Protocol=0-shot2025.10 | 37.8 | |
| D1Sequence Length=512, Evaluation Protocol=0-shot2025.10 | 37.2 | |
| LLaDA-8B-Inst.Sequence Length=256, Evaluation Protocol=0-shot2025.10 | 35.4 | |
| LLaDA-8B-BaseNumber of shots=02026.06 | 35.4 | |
| LLaDA-8B-Inst.Sequence Length=512, Evaluation Protocol=0-shot2025.10 | 34.8 | |
| LLaDA-1.5Sequence Length=256, Evaluation Protocol=0-shot2025.10 | 34.8 | |
| SPG w/ EUBOSequence Length=512, Evaluation Protocol=0-shot2025.10 | 33.5 | |
| LLaDA-1.5Sequence Length=512, Evaluation Protocol=0-shot2025.10 | 32.3 | |
| SPG w/ MixtureSequence Length=128, Evaluation Protocol=0-shot2025.10 | 31.1 | |
| D1Sequence Length=128, Evaluation Protocol=0-shot2025.10 | 29.3 | |
| SPG w/ EUBOSequence Length=128, Evaluation Protocol=0-shot2025.10 | 29.3 | |
| LLaDA-8B-Inst.Sequence Length=128, Evaluation Protocol=0-shot2025.10 | 28.1 | |
| UniGRPOSequence Length=128, Evaluation Protocol=0-shot2025.10 | 28.1 | |
| WD1Sequence Length=128, Evaluation Protocol=0-shot2025.10 | 25.6 | |
| LLaDA-1.5Sequence Length=128, Evaluation Protocol=0-shot2025.10 | 22.6 |