Multitask Language Understanding on MMLU f_con o=5
99.5Accuracy (Exact)W/O Decontamination
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| W/O DecontaminationModel=Mistral, Access=–2026.01 | 99.5 | 54.4 | 97.2 | 53.8 | 46.3 | 5.7 | |
| ITDModel=Mistral, Access=Black-box2026.01 | 99.4 | 54.3 | 96.7 | 53.3 | 46.7 | 6.1 | |
| W/O DecontaminationModel=LLaMA-3, Access=–2026.01 | 99.2 | 41.6 | 96.1 | 39.5 | 64.1 | 9.2 | |
| ITDModel=Qwen 2.5, Access=Black-box2026.01 | 98.3 | 31.3 | 93.8 | 26.7 | 75.2 | 7.9 | |
| W/O DecontaminationModel=Qwen 2.5, Access=–2026.01 | 98.1 | 31.1 | 94.2 | 27.1 | 74 | 6.7 | |
| TEDModel=Qwen 2.5, Access=Black-box2026.01 | 87 | 20 | 83.7 | 16.6 | 88.6 | 21.3 | |
| ITDModel=LLaMA-3, Access=Black-box2026.01 | 82.4 | 24.8 | 82.8 | 26.3 | 61 | 6.1 | |
| DeconIEPModel=Qwen 2.5, Access=White-box2026.01 | 80.5 | 13.5 | 72.5 | 5.4 | 64.1 | 3.2 | |
| TEDModel=LLaMA-3, Access=Black-box2026.01 | 79.4 | 21.8 | 78.5 | 22 | 61 | 6.1 | |
| Short CircuitModel=Qwen 2.5, Access=White-box2026.01 | 75 | 8 | 75.3 | 8.2 | 70.7 | 3.4 | |
| DeconIEPModel=LLaMA-3, Access=White-box2026.01 | 73.6 | 16 | 52.7 | 3.8 | 52.7 | 2.2 | |
| Shortcut NeuronModel=LLaMA-3, Access=White-box2026.01 | 70.5 | 13 | 68.5 | 12 | 42.7 | 12.2 | |
| Short CircuitModel=LLaMA-3, Access=White-box2026.01 | 69.7 | 12.1 | 69.5 | 13 | 70.6 | 15.7 | |
| Short CircuitModel=Mistral, Access=White-box2026.01 | 58.1 | 13 | 53.5 | 10.1 | 40.7 | 0.1 | |
| DeconIEPModel=Mistral, Access=White-box2026.01 | 45.4 | 0.3 | 42.3 | 1.1 | 43.3 | 2.7 | |
| Shortcut NeuronModel=Qwen 2.5, Access=White-box2026.01 | 40.5 | 26.5 | 41.9 | 25.2 | 35.9 | 31.4 | |
| Shortcut NeuronModel=Mistral, Access=White-box2026.01 | 39.4 | 5.7 | 40.9 | 2.5 | 24.5 | 16.1 | |
| TEDModel=Mistral, Access=Black-box2026.01 | 7.4 | 37.7 | 32.4 | 11 | 38.3 | 2.3 |