Multi-task Language Understanding on MMLU (Generic Score)
63.73MMLU ScoreMONA
Evaluation Results
| Method | Links | |
|---|---|---|
| MONAModel=MOE-68B-A3B, Training Tokens=700B, Evaluation Mode=FewShot2026.05 | 63.73 | |
| MuonModel=MOE-68B-A3B, Training Tokens=700B, Evaluation Mode=FewShot2026.05 | 62.81 | |
| AdamWModel=MOE-68B-A3B, Training Tokens=700B, Evaluation Mode=FewShot2026.05 | 62.18 | |
| Llama-8B (unpruned)Model Backbone=Llama-8B, Model Variant=unpruned, Global Sparsity Level=0%2026.05 | 0.667 | |
| Llama-8B (unpruned)Model Backbone=Llama-8B, Model Variant=unpruned, Global Sparsity Level=0%2026.05 | 0.667 | |
| Qwen-3B (unpruned)Model Backbone=Qwen-3B, Model Variant=unpruned, Global Sparsity Level=0%2026.05 | 0.644 | |
| Qwen-3B (unpruned)Model Backbone=Qwen-3B, Model Variant=unpruned, Global Sparsity Level=0%2026.05 | 0.644 | |
| Qwen-3B (KOFF)Model Backbone=Qwen-3B, Model Variant=KOFF, Global Sparsity Level=12%2026.05 | 0.61 | |
| Llama-8B (KOFF)Model Backbone=Llama-8B, Model Variant=KOFF, Global Sparsity Level=12%2026.05 | 0.603 | |
| Llama-3B (unpruned)Model Backbone=Llama-3B, Model Variant=unpruned, Global Sparsity Level=0%2026.05 | 0.56 | |
| Llama-3B (unpruned)Model Backbone=Llama-3B, Model Variant=unpruned, Global Sparsity Level=0%2026.05 | 0.56 | |
| Llama-3B (KOFF)Model Backbone=Llama-3B, Model Variant=KOFF, Global Sparsity Level=12%2026.05 | 0.52 | |
| Llama-3B (KOFF)Model Backbone=Llama-3B, Model Variant=KOFF, Global Sparsity Level=12%2026.05 | 0.484 | |
| Llama-8B (KOFF)Model Backbone=Llama-8B, Model Variant=KOFF, Global Sparsity Level=12%2026.05 | 0.415 | |
| Qwen-3B (KOFF)Model Backbone=Qwen-3B, Model Variant=KOFF, Global Sparsity Level=12%2026.05 | 0.413 | |
| Llama-8B (pruning only)Model Backbone=Llama-8B, Model Variant=pruning only, Global Sparsity Level=12%2026.05 | 0.408 | |
| QVecInference Precision=NF42026.06 | 0.396 | |
| AttackedInference Precision=LLM.int8()2026.06 | 0.391 | |
| OriginalInference Precision=FP322026.06 | 0.39 | |
| AttackedInference Precision=NF42026.06 | 0.388 | |
| GaussianInference Precision=LLM.int8()2026.06 | 0.387 | |
| Llama-8B (pruning only)Model Backbone=Llama-8B, Model Variant=pruning only, Global Sparsity Level=12%2026.05 | 0.384 | |
| OriginalInference Precision=FP322026.06 | 0.364 | |
| QVecInference Precision=LLM.int8()2026.06 | 0.361 | |
| OriginalInference Precision=FP322026.06 | 0.36 | |
| GaussianInference Precision=NF42026.06 | 0.335 | |
| AttackedInference Precision=FP42026.06 | 0.333 | |
| Qwen-3B (pruning only)Model Backbone=Qwen-3B, Model Variant=pruning only, Global Sparsity Level=12%2026.05 | 0.329 | |
| GaussianInference Precision=FP42026.06 | 0.328 | |
| Qwen-3B (pruning only)Model Backbone=Qwen-3B, Model Variant=pruning only, Global Sparsity Level=12%2026.05 | 0.326 | |
| Llama-3B (pruning only)Model Backbone=Llama-3B, Model Variant=pruning only, Global Sparsity Level=12%2026.05 | 0.316 | |
| QVecInference Precision=FP42026.06 | 0.301 | |
| Llama-3B (pruning only)Model Backbone=Llama-3B, Model Variant=pruning only, Global Sparsity Level=12%2026.05 | 0.249 |