Multi-task Language Understanding on MMLU (Calibration Metrics)
0.087Error Rate (ER)Seq. Post.
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Seq. Post.LLM=Gemini 2.5 Pro2026.05 | 0.087 | 0.179 | 62.2 | 0.545 | 12.8 | 10.8 | 0.416 | 21.5 | 9.5 | |
| Is TrueLLM=Gemini 2.5 Pro2026.05 | 0.087 | 0.086 | 52.3 | 0.653 | 8.7 | 10.2 | 0.653 | 17.4 | 8.8 | |
| VerbalizedLLM=Gemini 2.5 Pro2026.05 | 0.095 | 0.037 | 66.9 | 0.455 | 8.5 | 5.8 | 0.461 | 18 | 9.6 | |
| VerbalizedLLM=Gemini 2.5 Flash2026.05 | 0.126 | 0.094 | 64.6 | 1.054 | 11.4 | 7.3 | 1.059 | 24 | 12.7 | |
| Seq. Post.LLM=Gemini 2.5 Flash2026.05 | 0.145 | 0.131 | 72.6 | 1.003 | 13.5 | 8.5 | 0.979 | 28 | 14.8 | |
| Is TrueLLM=Gemini 2.5 Flash2026.05 | 0.145 | 0.135 | 54.7 | 1.346 | 14.3 | 13.8 | 1.347 | 28.8 | 14.6 | |
| Seq. Post.LLM=Gemini 2.5 Flash Lite2026.05 | 0.181 | 0.083 | 81.6 | 0.422 | 12.7 | 5.9 | 0.455 | 30.8 | 18.2 | |
| Is TrueLLM=Gemini 2.5 Flash Lite2026.05 | 0.181 | 0.152 | 52.7 | 1.07 | 17.2 | 19.1 | 1.074 | 35.3 | 18.2 | |
| VerbalizedLLM=Gemini 2.5 Flash Lite2026.05 | 0.187 | 0.104 | 62.4 | 0.599 | 15.6 | 12.3 | 0.616 | 34.3 | 18.8 | |
| Seq. Post.LLM=Qwen3.5 9B2026.05 | 0.234 | 0.046 | 84 | 0.409 | 13.1 | 7.2 | 0.474 | 36.5 | 23.6 | |
| Is TrueLLM=Qwen3.5 9B2026.05 | 0.234 | 0.175 | 52.6 | 0.793 | 21.1 | 21.5 | 0.804 | 44.5 | 23.6 | |
| VerbalizedLLM=Qwen3.5 9B2026.05 | 0.234 | 0.185 | 61.1 | 1.724 | 20.7 | 18.5 | 1.711 | 44.1 | 23.8 | |
| Seq. Post.LLM=GLM-4.6V-Flash2026.05 | 0.264 | 0.151 | 80.9 | 0.591 | 18.2 | 10.3 | 0.64 | 44.6 | 26.6 | |
| Is TrueLLM=GLM-4.6V-Flash2026.05 | 0.264 | 0.193 | 50.2 | 0.671 | 23.9 | 54.4 | 0.669 | 50.3 | 26.6 | |
| Seq. Post.LLM=Qwen3.5 4B2026.05 | 0.265 | 0.03 | 82.4 | 0.442 | 14.6 | 9.3 | 0.516 | 41.1 | 26.7 | |
| Is TrueLLM=Qwen3.5 4B2026.05 | 0.265 | 0.092 | 49.4 | 0.623 | 20.7 | 27.2 | 0.654 | 47.2 | 26.7 | |
| VerbalizedLLM=Qwen3.5 4B2026.05 | 0.273 | 0.217 | 60.4 | 2.049 | 23.9 | 22.2 | 2.053 | 51.2 | 27.6 | |
| VerbalizedLLM=GLM-4.6V-Flash2026.05 | 0.28 | 0.202 | 57 | 1.193 | 24 | 23 | 1.157 | 52 | 28.6 | |
| VerbalizedLLM=Ministral-3-8B-Instruct-25122026.05 | 0.327 | 0.234 | 56.5 | 1.148 | 27.2 | 29.9 | 1.049 | 59.9 | 33.8 | |
| Seq. Post.LLM=Ministral-3-8B-Instruct-25122026.05 | 0.372 | 0.316 | 54.1 | 0.889 | 33.3 | 45.8 | 0.821 | 70.5 | 37.5 | |
| Is TrueLLM=Ministral-3-8B-Instruct-25122026.05 | 0.372 | 0.141 | 50.5 | 0.771 | 26.4 | 37.8 | 0.832 | 63.6 | 37.5 |