Multi-task Language Understanding on MMLU (Accuracy, ECE, AURC, Spearman)
2.5ECEORCE
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ORCEFoundation Model=LLAMA-3 8B2026.05 | 2.5 | 0.477 | 0 | 15.6 | 8.9 | 65.7 | |
| ORCEFoundation Model=Qwen3 8B2026.05 | 3.4 | 0.414 | 0 | 11.6 | 8 | 74.9 | |
| Top-kFoundation Model=Qwen3 8B2026.05 | 4.3 | 0.247 | 8.568 | 17.7 | 13.6 | 72.6 | |
| SFTFoundation Model=Mistral 7B2026.05 | 6.6 | 0.316 | 0 | 22.8 | 14.7 | 62.5 | |
| SFTFoundation Model=LLAMA-3 8B2026.05 | 6.8 | 0.433 | 0 | 16.5 | 9.8 | 65.7 | |
| ORCEFoundation Model=Mistral 7B2026.05 | 7.3 | 0.408 | 0 | 20.6 | 12.4 | 62.5 | |
| ConfTunerFoundation Model=Mistral 7B2026.05 | 8.4 | 0.348 | 0 | 22.4 | 14.3 | 62.5 | |
| ConfTunerFoundation Model=Qwen3 8B2026.05 | 8.8 | 0.4 | 0 | 12.1 | 8.5 | 74.9 | |
| SFTFoundation Model=Qwen3 8B2026.05 | 13 | 0.39 | 0 | 14.9 | 11.3 | 74.9 | |
| CoTFoundation Model=Qwen3 8B2026.05 | 13.7 | 0.393 | 0 | 14 | 10.2 | 73.8 | |
| ConfTunerFoundation Model=LLAMA-3 8B2026.05 | 16.7 | 0.409 | 0 | 17.2 | 10.5 | 65.7 | |
| VanillaFoundation Model=LLAMA-3 8B2026.05 | 17 | 0.034 | 4.682 | 33.2 | 26.5 | 65.7 | |
| Top-kFoundation Model=LLAMA-3 8B2026.05 | 18.1 | 0.149 | 1.236 | 29.9 | 22.4 | 63.9 | |
| ConsistencyFoundation Model=LLAMA-3 8B2026.05 | 18.5 | 0.464 | 0 | 19.6 | 12.7 | 65.3 | |
| ConsistencyFoundation Model=Qwen3 8B2026.05 | 20.4 | 0.368 | 0 | 16.3 | 12.8 | 74.5 | |
| VanillaFoundation Model=Qwen3 8B2026.05 | 21.2 | 0.198 | 1.143 | 17.5 | 13.9 | 74.9 | |
| CoTFoundation Model=LLAMA-3 8B2026.05 | 23.2 | 0.204 | 3.058 | 31.3 | 23.1 | 62.4 | |
| Top-kFoundation Model=Mistral 7B2026.05 | 24.7 | 0.171 | 8.971 | 34.7 | 26.1 | 61.4 | |
| VanillaFoundation Model=Mistral 7B2026.05 | 24.8 | 0.01 | 0.24 | 37.4 | 29.2 | 62.5 | |
| ConsistencyFoundation Model=Mistral 7B2026.05 | 27.4 | 0.371 | 0 | 27.5 | 19.3 | 62.3 | |
| CoTFoundation Model=Mistral 7B2026.05 | 28.4 | 0.126 | 5.432 | 38.5 | 28.3 | 58.4 |