Multitask Language Understanding on MMLU (Exact Split, o=3)
92.1AccuracyW/O Decontamination
Evaluation results
| Method | Links | ||
|---|---|---|---|
| W/O DecontaminationModel=LLaMA-3, Split=f_con2026.01 | 92.1 | 0.345 | |
| W/O DecontaminationModel=Qwen2.5, Split=f_con2026.01 | 90.5 | 0.235 | |
| ITDModel=Qwen2.5, Split=f_con, Access=Black-box2026.01 | 89.8 | 0.228 | |
| W/O DecontaminationModel=Mistral, Split=f_con2026.01 | 89.2 | 0.441 | |
| ITDModel=Mistral, Split=f_con, Access=Black-box2026.01 | 88.6 | 0.435 | |
| ITDModel=LLaMA-3, Split=f_con, Access=Black-box2026.01 | 88.6 | 0.31 | |
| TEDModel=LLaMA-3, Split=f_con, Access=Black-box2026.01 | 86.4 | 0.288 | |
| TEDModel=Qwen2.5, Split=f_con, Access=Black-box2026.01 | 83.7 | 0.167 | |
| Short CircuitModel=Mistral, Split=f_con, Access=White-box2026.01 | 83.3 | 0.382 | |
| Short CircuitModel=Qwen2.5, Split=f_con, Access=White-box2026.01 | 80.7 | 0.137 | |
| DeconIEPModel=Qwen2.5, Split=f_con, Access=White-box2026.01 | 78.5 | 0.115 | |
| ITDModel=Qwen2.5, Split=f_con, Access=Black-box2026.01 | 75.1 | 0.078 | |
| W/O DecontaminationModel=Qwen2.5, Split=f_con2026.01 | 74.4 | 0.071 | |
| DeconIEPModel=LLaMA-3, Split=f_con, Access=White-box2026.01 | 74.1 | 0.165 | |
| TEDModel=Qwen2.5, Split=f_con, Access=Black-box2026.01 | 71.3 | 0.04 | |
| Short CircuitModel=LLaMA-3, Split=f_con, Access=White-box2026.01 | 70.7 | 0.157 | |
| Short CircuitModel=LLaMA-3, Split=f_con, Access=White-box2026.01 | 70.1 | 0.125 | |
| Shortcut NeuronModel=LLaMA-3, Split=f_con, Access=White-box2026.01 | 69 | 0.114 | |
| baselineModel=Qwen2.5, Split=f_clean2026.01 | 67.3 | — | |
| baselineModel=Qwen2.5, Split=f_clean2026.01 | 67 | — | |
| W/O DecontaminationModel=LLaMA-3, Split=f_con2026.01 | 66.9 | 0.12 | |
| ITDModel=LLaMA-3, Split=f_con, Access=Black-box2026.01 | 64.8 | 0.098 | |
| DeconIEPModel=Qwen2.5, Split=f_con, Access=White-box2026.01 | 64.6 | 0.027 | |
| Short CircuitModel=Qwen2.5, Split=f_con, Access=White-box2026.01 | 64 | 0.033 | |
| TEDModel=LLaMA-3, Split=f_con, Access=Black-box2026.01 | 63.8 | 0.089 | |
| baselineModel=LLaMA-3, Split=f_clean2026.01 | 57.6 | — | |
| baselineModel=LLaMA-3, Split=f_clean2026.01 | 54.9 | — | |
| DeconIEPModel=LLaMA-3, Split=f_con, Access=White-box2026.01 | 53.6 | 0.014 | |
| Shortcut NeuronModel=LLaMA-3, Split=f_con, Access=White-box2026.01 | 48.2 | 0.068 | |
| Short CircuitModel=Mistral, Split=f_con, Access=White-box2026.01 | 47.8 | 0.072 | |
| W/O DecontaminationModel=Mistral, Split=f_con2026.01 | 47.5 | 0.069 | |
| ITDModel=Mistral, Split=f_con, Access=Black-box2026.01 | 47.1 | 0.065 | |
| DeconIEPModel=Mistral, Split=f_con, Access=White-box2026.01 | 45.8 | 0.007 | |
| baselineModel=Mistral, Split=f_clean2026.01 | 45.1 | — | |
| DeconIEPModel=Mistral, Split=f_con, Access=White-box2026.01 | 41.7 | 0.011 | |
| Shortcut NeuronModel=Qwen2.5, Split=f_con, Access=White-box2026.01 | 41.6 | 0.254 | |
| baselineModel=Mistral, Split=f_clean2026.01 | 40.6 | — | |
| Shortcut NeuronModel=Mistral, Split=f_con, Access=White-box2026.01 | 38 | 0.071 | |
| TEDModel=Mistral, Split=f_con, Access=Black-box2026.01 | 37.5 | 0.031 | |
| Shortcut NeuronModel=Qwen2.5, Split=f_con, Access=White-box2026.01 | 37.1 | 0.302 | |
| Shortcut NeuronModel=Mistral, Split=f_con, Access=White-box2026.01 | 23.9 | 0.167 | |
| TEDModel=Mistral, Split=f_con, Access=Black-box2026.01 | 22.4 | 0.227 |