Multi-task Language Understanding on MMLU (Zero-shot Accuracy)
79.95Zero-shot AccuracyNemotron-Labs-Diffusion-8B
Evaluation Results
| Method | Links | |
|---|---|---|
| Nemotron-Labs-Diffusion-8BGen. Mode=Quad. SS, TPF=6.382026.07 | 79.95 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=AR, TPF=1.002026.07 | 79.85 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Linear SS, TPF=5.992026.07 | 79.01 | |
| SDAR-8B ChatGen. Mode=Diff., TPF=1.752026.07 | 78.83 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Diff., TPF=2.572026.07 | 78.71 | |
| SDAR-8B ChatGen. Mode=Diff., TPF=1.002026.07 | 78.6 | |
| Qwen3 8BGen. Mode=AR, TPF=1.002026.07 | 76.66 | |
| Qwen2.5 7BGen. Mode=AR, TPF=1.002026.07 | 74.86 | |
| Ministral3-8B Instruct-2512Gen. Mode=AR, TPF=1.002026.07 | 73.9 | |
| Dream-7B InstructGen. Mode=Diff., TPF=1.002026.07 | 67 | |
| LLaDA-8B InstructGen. Mode=Diff., TPF=1.002026.07 | 65.5 | |
| DLR + CoLAEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.06 | 22.95 | |
| CoLAEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.06 | 22.95 | |
| Full-RankEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.06 | 22.95 |