Language Understanding on MMLU (o=1 Exact Split)
77.6AccuracyITD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ITDModel=Qwen 2.5, Split=fcon, Access=Black-box2026.01 | 77.6 | 0.106 | |
| W/O DecontaminationModel=Qwen 2.5, Split=fcon, Access=-2026.01 | 77.5 | 0.105 | |
| TEDModel=Qwen 2.5, Split=fcon, Access=Black-box2026.01 | 74.2 | 0.072 | |
| ITDModel=Qwen 2.5, Split=fcon, Access=Black-box2026.01 | 74.2 | 0.069 | |
| W/O DecontaminationModel=Qwen 2.5, Split=fcon, Access=-2026.01 | 73.5 | 0.062 | |
| W/O DecontaminationModel=LLaMA-3, Split=fcon, Access=-2026.01 | 73.4 | 0.159 | |
| DeconIEPModel=Qwen 2.5, Split=fcon, Access=White-box2026.01 | 73 | 0.057 | |
| TEDModel=LLaMA-3, Split=fcon, Access=Black-box2026.01 | 72.7 | 0.151 | |
| ITDModel=LLaMA-3, Split=fcon, Access=Black-box2026.01 | 72.1 | 0.145 | |
| Short CircuitModel=LLaMA-3, Split=fcon, Access=White-box2026.01 | 71 | 0.16 | |
| DeconIEPModel=Qwen 2.5, Split=fcon, Access=White-box2026.01 | 70.5 | 0.035 | |
| Short CircuitModel=LLaMA-3, Split=fcon, Access=White-box2026.01 | 69.5 | 0.12 | |
| baselineModel=Qwen 2.5, Split=fclean, Access=-2026.01 | 67.3 | — | |
| baselineModel=Qwen 2.5, Split=fclean, Access=-2026.01 | 67 | — | |
| W/O DecontaminationModel=LLaMA-3, Split=fcon, Access=-2026.01 | 66.3 | 0.114 | |
| Short CircuitModel=Mistral, Split=fcon, Access=White-box2026.01 | 63.1 | 0.18 | |
| ITDModel=LLaMA-3, Split=fcon, Access=Black-box2026.01 | 62 | 0.07 | |
| Short CircuitModel=Qwen 2.5, Split=fcon, Access=White-box2026.01 | 61.9 | 0.051 | |
| TEDModel=Qwen 2.5, Split=fcon, Access=Black-box2026.01 | 61.9 | 0.054 | |
| W/O DecontaminationModel=Mistral, Split=fcon, Access=-2026.01 | 61.7 | 0.166 | |
| DeconIEPModel=LLaMA-3, Split=fcon, Access=White-box2026.01 | 61.6 | 0.04 | |
| ITDModel=Mistral, Split=fcon, Access=Black-box2026.01 | 61 | 0.159 | |
| Short CircuitModel=Qwen 2.5, Split=fcon, Access=White-box2026.01 | 61 | 0.063 | |
| Short CircuitModel=Mistral, Split=fcon, Access=White-box2026.01 | 59.4 | 0.188 | |
| baselineModel=LLaMA-3, Split=fclean, Access=-2026.01 | 57.6 | — | |
| TEDModel=LLaMA-3, Split=fcon, Access=Black-box2026.01 | 57.6 | 0.027 | |
| baselineModel=LLaMA-3, Split=fclean, Access=-2026.01 | 54.9 | — | |
| Shortcut NeuronModel=LLaMA-3, Split=fcon, Access=White-box2026.01 | 54.7 | 0.029 | |
| DeconIEPModel=LLaMA-3, Split=fcon, Access=White-box2026.01 | 51.5 | 0.034 | |
| ITDModel=Mistral, Split=fcon, Access=Black-box2026.01 | 49.9 | 0.093 | |
| W/O DecontaminationModel=Mistral, Split=fcon, Access=-2026.01 | 49.2 | 0.086 | |
| Shortcut NeuronModel=LLaMA-3, Split=fcon, Access=White-box2026.01 | 48.4 | 0.065 | |
| DeconIEPModel=Mistral, Split=fcon, Access=White-box2026.01 | 47.8 | 0.027 | |
| DeconIEPModel=Mistral, Split=fcon, Access=White-box2026.01 | 47 | 0.064 | |
| baselineModel=Mistral, Split=fclean, Access=-2026.01 | 45.1 | — | |
| baselineModel=Mistral, Split=fclean, Access=-2026.01 | 40.6 | — | |
| TEDModel=Mistral, Split=fcon, Access=Black-box2026.01 | 40.4 | 0.047 | |
| Shortcut NeuronModel=Qwen 2.5, Split=fcon, Access=White-box2026.01 | 39.5 | 0.275 | |
| Shortcut NeuronModel=Qwen 2.5, Split=fcon, Access=White-box2026.01 | 38.9 | 0.284 | |
| TEDModel=Mistral, Split=fcon, Access=Black-box2026.01 | 36.3 | 0.043 | |
| Shortcut NeuronModel=Mistral, Split=fcon, Access=White-box2026.01 | 30 | 0.151 | |
| Shortcut NeuronModel=Mistral, Split=fcon, Access=White-box2026.01 | 26.3 | 0.143 |