MMLU Question Answering (Accuracy and Performance Gain)
78.7AccuracyMistral Small 24B Inst 2501
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Mistral Small 24B Inst 2501Classifier=Self-labeled2026.01 | 78.7 | -0.6 | |
| Mistral Small 24B Inst 2501Classifier=Majority-labeled2026.01 | 78.5 | -0.8 | |
| BaselineBackbone=Qwen3-MoE-30B-A3B, Type=–, Storage=61.06GB, Evaluation Protocol=5-shot2026.06 | 77.8 | — | |
| Mistral Small 24B Base 2501Classifier=Self-labeled2026.01 | 76.7 | -1.5 | |
| Mistral Small 24B Base 2501Classifier=Majority-labeled2026.01 | 76.6 | -1.6 | |
| PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Type=M25%, Storage=46.56GB, Evaluation Protocol=5-shot2026.06 | 76.6 | — | |
| PuzzleMoEBackbone=Qwen3-MoE-30B-A3B, Type=M50%, Storage=32.07GB, Evaluation Protocol=5-shot2026.06 | 75.1 | — | |
| OursQBackbone=Qwen3-MoE-30B-A3B, Type=P25% Q4b, Storage=11.64GB, Evaluation Protocol=5-shot2026.06 | 73.04 | — | |
| Llama 3.1 8B InstClassifier=Self-labeled2026.01 | 65.3 | -0.1 | |
| Llama 3.1 8B InstClassifier=Majority-labeled2026.01 | 65.3 | -0.1 | |
| Llama 3.1 8BClassifier=Self-labeled2026.01 | 63.9 | -0.8 | |
| Llama 3.1 8BClassifier=Majority-labeled2026.01 | 63.6 | -1.1 | |
| Mistral Nemo Base 2407Classifier=Self-labeled2026.01 | 62.6 | -1.3 | |
| Mistral Nemo Base 2407Classifier=Majority-labeled2026.01 | 62.3 | -1.6 | |
| OursBackbone=Qwen3-MoE-30B-A3B, Type=P50%, Storage=32.07GB, Evaluation Protocol=5-shot2026.06 | 61.27 | — | |
| BaselineBackbone=Qwen1.5-MoE-A2.7B, Type=–, Storage=28.67GB, Evaluation Protocol=5-shot2026.06 | 61.08 | — | |
| PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Type=M25%, Storage=22.40GB, Evaluation Protocol=5-shot2026.06 | 60.4 | — | |
| WandaBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=7.10GB, Evaluation Protocol=5-shot2026.06 | 60.34 | — | |
| Mistral Nemo Inst 2407Classifier=Self-labeled2026.01 | 60.2 | 1 | |
| PuzzleMoEBackbone=Qwen1.5-MoE-A2.7B, Type=M50%, Storage=16.17GB, Evaluation Protocol=5-shot2026.06 | 60 | — | |
| Mistral 7B Inst v0.3Classifier=Self-labeled2026.01 | 59.3 | -1 | |
| Mistral 7B v0.3Classifier=Self-labeled2026.01 | 58.7 | -1.3 | |
| Mistral 7B Inst v0.3Classifier=Majority-labeled2026.01 | 58.7 | -1.6 | |
| Mistral Nemo Inst 2407Classifier=Majority-labeled2026.01 | 58.7 | -0.4 | |
| Llama 3.2 3B InstClassifier=Self-labeled2026.01 | 58.5 | -0.1 | |
| Mistral 7B v0.3Classifier=Majority-labeled2026.01 | 58.5 | -1.5 | |
| Llama 3.2 3B InstClassifier=Majority-labeled2026.01 | 58.4 | -0.2 | |
| OursQBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=5-shot2026.06 | 58.29 | — | |
| WandaBackbone=Qwen1.5-MoE-A2.7B, Type=P25%, Storage=28.67GB, Evaluation Protocol=5-shot2026.06 | 57.55 | — | |
| C-PruneBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=5-shot2026.06 | 56.68 | — | |
| MoNEBackbone=Qwen1.5-MoE-A2.7B, Type=P25%, Storage=22.40GB, Evaluation Protocol=5-shot2026.06 | 56.19 | — | |
| EAC-MoEBackbone=Qwen1.5-MoE-A2.7B, Type=P38% Q3.03b, Storage=4.35GB, Evaluation Protocol=5-shot2026.06 | 55.96 | — | |
| Llama 3.2 3BClassifier=Self-labeled2026.01 | 55.9 | -1.1 | |
| Llama 3.2 3BClassifier=Majority-labeled2026.01 | 55.4 | -1.6 | |
| MoNEBackbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=5-shot2026.06 | 55.39 | — | |
| WandaBackbone=Qwen1.5-MoE-A2.7B, Type=P50%, Storage=28.67GB, Evaluation Protocol=5-shot2026.06 | 53.82 | — | |
| He et al.Backbone=Qwen1.5-MoE-A2.7B, Type=P25% Q4b, Storage=5.60GB, Evaluation Protocol=5-shot2026.06 | 52.73 | — | |
| OursBackbone=Qwen1.5-MoE-A2.7B, Type=P50%, Storage=16.17GB, Evaluation Protocol=5-shot2026.06 | 50.67 | — | |
| Llama 3.2 1B InstClassifier=Self-labeled2026.01 | 45.9 | 0.1 | |
| MoNEBackbone=Qwen1.5-MoE-A2.7B, Type=P50%, Storage=16.17GB, Evaluation Protocol=5-shot2026.06 | 45.67 | — | |
| Llama 3.2 1B InstClassifier=Majority-labeled2026.01 | 45.6 | -0.2 | |
| Llama 3.2 1BClassifier=Self-labeled2026.01 | 43.8 | 1.5 | |
| Llama 3.2 1BClassifier=Majority-labeled2026.01 | 41.1 | -1.1 | |
| He et al.Backbone=Qwen1.5-MoE-A2.7B, Type=P50%, Storage=16.17GB, Evaluation Protocol=5-shot2026.06 | 31.52 | — | |
| WandaBackbone=Qwen3-MoE-30B-A3B, Type=P25%, Storage=61.06GB, Evaluation Protocol=5-shot2026.06 | 22.95 | — | |
| WandaBackbone=Qwen3-MoE-30B-A3B, Type=P50%, Storage=61.06GB, Evaluation Protocol=5-shot2026.06 | 22.93 | — |