List your company
ResearchFormerly WizWand

Multi-task Language Understanding on MMLU (Zero-shot Accuracy)

79.95Zero-shot AccuracyNemotron-Labs-Diffusion-8B
20.6736.0651.4566.84Jun 27, 2026Jun 28, 2026Jun 30, 2026Jul 2, 2026Jul 3, 2026Jul 5, 2026Jul 7, 2026
Updated 3mo ago

Evaluation results

MethodLinks
Nemotron-Labs-Diffusion-8BGen. Mode=Quad. SS, TPF=6.382026.0779.95
Nemotron-Labs-Diffusion-8BGen. Mode=AR, TPF=1.002026.0779.85
Nemotron-Labs-Diffusion-8BGen. Mode=Linear SS, TPF=5.992026.0779.01
SDAR-8B ChatGen. Mode=Diff., TPF=1.752026.0778.83
Nemotron-Labs-Diffusion-8BGen. Mode=Diff., TPF=2.572026.0778.71
SDAR-8B ChatGen. Mode=Diff., TPF=1.002026.0778.6
Qwen3 8BGen. Mode=AR, TPF=1.002026.0776.66
Qwen2.5 7BGen. Mode=AR, TPF=1.002026.0774.86
Ministral3-8B Instruct-2512Gen. Mode=AR, TPF=1.002026.0773.9
Dream-7B InstructGen. Mode=Diff., TPF=1.002026.0767
LLaDA-8B InstructGen. Mode=Diff., TPF=1.002026.0765.5
DLR + CoLAEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.0622.95
CoLAEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.0622.95
Full-RankEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.0622.95