Multitask Language Understanding on MMLU (Semantic-level split, o=3)
90.1AccuracyW/O Decontamination
Evaluation results
| Method | Links | ||
|---|---|---|---|
| W/O DecontaminationModel=LLaMA-3, Split=f_con2026.01 | 90.1 | 33.6 | |
| TEDModel=Qwen2.5, Split=f_con, Access=Black-box2026.01 | 88.2 | 21.1 | |
| W/O DecontaminationModel=Mistral, Split=f_con2026.01 | 87.5 | 44.1 | |
| W/O DecontaminationModel=Qwen2.5, Split=f_con2026.01 | 87.2 | 20.1 | |
| ITDModel=Qwen2.5, Split=f_con, Access=Black-box2026.01 | 86.8 | 19.7 | |
| ITDModel=Mistral, Split=f_con, Access=Black-box2026.01 | 86.3 | 42.9 | |
| ITDModel=LLaMA-3, Split=f_con, Access=Black-box2026.01 | 84.4 | 27.9 | |
| TEDModel=LLaMA-3, Split=f_con, Access=Black-box2026.01 | 84.2 | 27.7 | |
| Short CircuitModel=Mistral, Split=f_con, Access=White-box2026.01 | 80.8 | 37.4 | |
| Short CircuitModel=Qwen2.5, Split=f_con, Access=White-box2026.01 | 75.7 | 8.6 | |
| Shortcut NeuronModel=LLaMA-3, Split=f_con, Access=White-box2026.01 | 68.2 | 11.7 | |
| Short CircuitModel=LLaMA-3, Split=f_con, Access=White-box2026.01 | 68 | 11.5 | |
| baselineModel=Qwen2.5, Split=f_clean2026.01 | 67.1 | — | |
| DeconIEPModel=Qwen2.5, Split=f_con, Access=White-box2026.01 | 63.2 | 3.9 | |
| baselineModel=LLaMA-3, Split=f_clean2026.01 | 56.5 | — | |
| DeconIEPModel=LLaMA-3, Split=f_con, Access=White-box2026.01 | 53.7 | 2.8 | |
| baselineModel=Mistral, Split=f_clean2026.01 | 43.4 | — | |
| Shortcut NeuronModel=Qwen2.5, Split=f_con, Access=White-box2026.01 | 42 | 25.1 | |
| DeconIEPModel=Mistral, Split=f_con, Access=White-box2026.01 | 40.6 | 2.8 | |
| TEDModel=Mistral, Split=f_con, Access=Black-box2026.01 | 38.8 | 4.6 | |
| Shortcut NeuronModel=Mistral, Split=f_con, Access=White-box2026.01 | 38.6 | 4.8 |