Question Answering on MMLU-Pro standard (test)
80.8AccuracyGPT-4.1
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4.1Approach=Standard2025.11 | 80.8 | — | — | — | |
| GPT-4.1Approach=Verbalized Conf.2025.11 | 80.6 | 71.5 | 0.172 | 0.172 | |
| GPT-4.1Approach=Verbalized Distrib.2025.11 | 80.2 | 81.2 | 0.124 | 0.045 | |
| Qwen3-30B-A3B-InstructApproach=Standard2025.11 | 79.7 | — | — | — | |
| Qwen3-30B-A3B-InstructApproach=Logit2025.11 | 79.7 | 73.4 | 0.191 | 0.19 | |
| Qwen3-30B-A3B-InstructApproach=p(True)2025.11 | 79.7 | 68.4 | 0.191 | 0.189 | |
| GPT-4.1Approach=Verbalized Top-k2025.11 | 79.6 | 79.8 | 0.141 | 0.092 | |
| Qwen3-30B-A3B-InstructApproach=Verbalized Conf.2025.11 | 79.4 | 72.4 | 0.179 | 0.162 | |
| Qwen3-30B-A3B-InstructApproach=Verbalized Top-k2025.11 | 79 | 75.5 | 0.172 | 0.152 | |
| DeepSeek-V3Approach=Verbalized Conf.2025.11 | 78.8 | 76.9 | 0.162 | 0.12 | |
| DeepSeek-V3Approach=Verbalized Distrib.2025.11 | 78.8 | 81.7 | 0.135 | 0.059 | |
| Qwen3-30B-A3B-InstructApproach=Verbalized Distrib.2025.11 | 78.6 | 77.7 | 0.14 | 0.059 | |
| DeepSeek-V3Approach=Verbalized Top-k2025.11 | 77.1 | 77.6 | 0.151 | 0.074 | |
| DeepSeek-V3Approach=Standard2025.11 | 77 | — | — | — | |
| Qwen3-4B-InstructApproach=Standard2025.11 | 72 | — | — | — | |
| Qwen3-4B-InstructApproach=Logit2025.11 | 72 | 74.4 | 0.261 | 0.259 | |
| Qwen3-4B-InstructApproach=p(True)2025.11 | 72 | 53.5 | 0.334 | 0.323 | |
| Qwen3-4B-InstructApproach=Verbalized Top-k2025.11 | 71.6 | 77.2 | 0.236 | 0.23 | |
| Qwen3-4B-InstructApproach=Verbalized Conf.2025.11 | 71.1 | 76.4 | 0.25 | 0.243 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Standard2025.11 | 70.9 | — | — | — | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Logit2025.11 | 70.9 | 56 | 0.287 | 0.288 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=p(True)2025.11 | 70.9 | 52 | 0.248 | 0.151 | |
| Qwen3-4B-InstructApproach=Verbalized Distrib.2025.11 | 70.7 | 77.3 | 0.174 | 0.091 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Verbalized Conf.2025.11 | 70.1 | 73.9 | 0.203 | 0.147 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Verbalized Top-k2025.11 | 70 | 72 | 0.21 | 0.156 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Verbalized Distrib.2025.11 | 69.8 | 73.4 | 0.185 | 0.07 | |
| Mistral-3.2-24B-InstructApproach=Standard2025.11 | 66.6 | — | — | — | |
| Mistral-3.2-24B-InstructApproach=Logit2025.11 | 66.6 | 65.8 | 0.299 | 0.294 | |
| Mistral-3.2-24B-InstructApproach=p(True)2025.11 | 66.6 | 73.5 | 0.196 | 0.092 | |
| Mistral-3.2-24B-InstructApproach=Verbalized Distrib.2025.11 | 65.9 | 72.3 | 0.209 | 0.129 | |
| Mistral-3.2-24B-InstructApproach=Verbalized Top-k2025.11 | 65.7 | 72.1 | 0.229 | 0.18 | |
| Mistral-3.2-24B-InstructApproach=Verbalized Conf.2025.11 | 64.8 | 70 | 0.263 | 0.242 |