Language Understanding on MMLU (Accuracy, Token Usage, and Delta Metrics)
77.37AccuracyDyMoE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DyMoEModel=Qwen3-30B-A3B, High / Low Bits=4 / 0, Retention ratio (r)=0.92026.03 | 77.37 | — | — | — | — | |
| DyMoEModel=Qwen3-30B-A3B, High / Low Bits=4 / 0, Retention ratio (r)=1.02026.03 | 77.05 | — | — | — | — | |
| DyMoEModel=Qwen3-30B-A3B, High / Low Bits=4 / 2, Retention ratio (r)=1.02026.03 | 77.05 | — | — | — | — | |
| DyMoEModel=Qwen3-30B-A3B, High / Low Bits=4 / 2, Retention ratio (r)=0.92026.03 | 77.04 | — | — | — | — | |
| DyMoEModel=Qwen3-30B-A3B, High / Low Bits=4 / 0, Retention ratio (r)=0.752026.03 | 76.98 | — | — | — | — | |
| DyMoEModel=Qwen3-30B-A3B, High / Low Bits=4 / 2, Retention ratio (r)=0.752026.03 | 76.32 | — | — | — | — | |
| TRiMSBackbone=DeepSeek-R1-Distill-Qwen-7B2026.03 | 72.9 | 316.6 | 14.4 | -72.7 | 230.3 | |
| DyMoEModel=Mixtral-8×7B, High / Low Bits=4 / 2, Retention ratio (r)=0.92026.03 | 68.21 | — | — | — | — | |
| DyMoEModel=Mixtral-8×7B, High / Low Bits=4 / 0, Retention ratio (r)=0.92026.03 | 68.07 | — | — | — | — | |
| DyMoEModel=Mixtral-8×7B, High / Low Bits=4 / 0, Retention ratio (r)=1.02026.03 | 67.95 | — | — | — | — | |
| DyMoEModel=Mixtral-8×7B, High / Low Bits=4 / 2, Retention ratio (r)=1.02026.03 | 67.95 | — | — | — | — | |
| DyMoEModel=Mixtral-8×7B, High / Low Bits=4 / 2, Retention ratio (r)=0.752026.03 | 67.82 | — | — | — | — | |
| DyMoEModel=Mixtral-8×7B, High / Low Bits=4 / 0, Retention ratio (r)=0.752026.03 | 67.14 | — | — | — | — | |
| VanillaBackbone=DeepSeek-R1-Distill-Qwen-7B2026.03 | 63.7 | 1,158.5 | — | — | 55 | |
| TRiMSBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.03 | 48.2 | 388.1 | 26.5 | -77.4 | 124.2 | |
| VanillaBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.03 | 38.1 | 1,718.7 | — | — | 22.2 | |
| AletheiaModel=Qwen2.5-3B2026.04 | — | — | -0.6 | — | — | |
| AletheiaModel=Qwen2.5-7B2026.04 | — | — | -0.54 | — | — | |
| AletheiaModel=Llama-3.1-8B2026.04 | — | — | 0.01 | — | — | |
| AletheiaModel=StableLM-2-1.6B2026.04 | — | — | -1.33 | — | — | |
| AletheiaModel=GPT-J-6B2026.04 | — | — | 0.5 | — | — | |
| AletheiaModel=Mixtral-8×7B2026.04 | — | — | -0.17 | — | — | |
| KICK-SETTLEDefense=SDD, Model=Llama-2-7B, Attack Type=Kick-settle2026.05 | — | — | 0.037 | — | — | |
| KICK-SETTLEDefense=SDD, Model=Qwen3-8B, Attack Type=Kick-settle2026.05 | — | — | 0.03 | — | — | |
| KICK-SETTLEDefense=SDD, Model=Llama-3.1-8B, Attack Type=Kick-settle2026.05 | — | — | 0.115 | — | — |