Multiple Choice Question Answering on MMLU (ACC, ECE, MRR, K-Tau, DRegret)
87.68AccuracyWALLA I
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| WALLA Icalibration=calibrated2026.07 | 87.68 | 8.21 | 70.71 | 41.02 | 9.99 | |
| WALLA Ipooling=log-pool2026.07 | 87.36 | 2.88 | 72.32 | 25.55 | 10.48 | |
| Gemma22026.07 | 87.18 | 8.72 | 82.8 | 32.61 | 12.94 | |
| StackedGen2026.07 | 87.09 | 1.99 | 82.24 | 42.17 | 10.6 | |
| WALLA II2026.07 | 86.98 | 3.01 | 82.52 | 40.42 | 11.7 | |
| NIRTRouter2026.07 | 86.91 | 2.15 | 82.72 | 43.72 | 11.51 | |
| RouteLLM2026.07 | 86.85 | 1.96 | 82.72 | 43.72 | 11.03 | |
| WALLA I2026.07 | 86.78 | 3.34 | 82.6 | 45.11 | 11.89 | |
| SelfCertainty2026.07 | 86.58 | 1.36 | 88.75 | 63.21 | 11.86 | |
| RouterDC2026.07 | 85.28 | 4.75 | 61.71 | 30.16 | 13.61 | |
| UniformAvg2026.07 | 85.15 | 5.1 | — | 0 | 13.87 | |
| PackLLM2026.07 | 84.94 | 5.13 | 37.79 | -27.81 | 14.28 | |
| Llama3.12026.07 | 81.99 | 2.52 | 40.88 | -8.01 | 17.61 | |
| Aloe2026.07 | 80.23 | 8.77 | 53.42 | 3.68 | 21.34 | |
| BioMistral2026.07 | 66.05 | 10.92 | 32.59 | -28.27 | 39.95 |