Question Answering on MedQA standard (test)
94AccuracyGPT-4.1
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4.1Approach=Verbalized Distrib.2025.11 | 94 | 81.2 | 0.056 | 0.073 | |
| GPT-4.1Approach=Verbalized Top-k2025.11 | 93.9 | 82.2 | 0.052 | 0.041 | |
| GPT-4.1Approach=Standard2025.11 | 93.6 | — | — | — | |
| GPT-4.1Approach=Verbalized Conf.2025.11 | 93.6 | 67.9 | 0.059 | 0.053 | |
| DeepSeek-V3Approach=Verbalized Conf.2025.11 | 88.8 | 75 | 0.089 | 0.029 | |
| DeepSeek-V3Approach=Verbalized Distrib.2025.11 | 88.3 | 80.7 | 0.097 | 0.096 | |
| DeepSeek-V3Approach=Verbalized Top-k2025.11 | 88.2 | 79.6 | 0.089 | 0.044 | |
| DeepSeek-V3Approach=Standard2025.11 | 88 | — | — | — | |
| Qwen3-30B-A3B-InstructApproach=Verbalized Top-k2025.11 | 87 | 65.5 | 0.112 | 0.072 | |
| Qwen3-30B-A3B-InstructApproach=Standard2025.11 | 86.9 | — | — | — | |
| Qwen3-30B-A3B-InstructApproach=Logit2025.11 | 86.9 | 68.3 | 0.123 | 0.121 | |
| Qwen3-30B-A3B-InstructApproach=p(True)2025.11 | 86.9 | 71.3 | 0.127 | 0.127 | |
| Qwen3-30B-A3B-InstructApproach=Verbalized Distrib.2025.11 | 86.7 | 68 | 0.118 | 0.096 | |
| Qwen3-30B-A3B-InstructApproach=Verbalized Conf.2025.11 | 86.6 | 67.9 | 0.117 | 0.088 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Verbalized Conf.2025.11 | 82.9 | 65.6 | 0.13 | 0.043 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Standard2025.11 | 82.3 | — | — | — | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Logit2025.11 | 82.3 | 66.5 | 0.176 | 0.176 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=p(True)2025.11 | 82.3 | 64.4 | 0.14 | 0.078 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Verbalized Distrib.2025.11 | 81.4 | 70 | 0.161 | 0.148 | |
| Baichuan-M2-32B-GPTQ-INT4Approach=Verbalized Top-k2025.11 | 79.1 | 70.4 | 0.151 | 0.029 | |
| Mistral-3.2-24B-InstructApproach=Standard2025.11 | 78.9 | — | — | — | |
| Mistral-3.2-24B-InstructApproach=Logit2025.11 | 78.9 | 69.7 | 0.194 | 0.196 | |
| Mistral-3.2-24B-InstructApproach=p(True)2025.11 | 78.9 | 77.7 | 0.147 | 0.083 | |
| Mistral-3.2-24B-InstructApproach=Verbalized Distrib.2025.11 | 78.5 | 69.9 | 0.155 | 0.052 | |
| Mistral-3.2-24B-InstructApproach=Verbalized Conf.2025.11 | 78 | 66.6 | 0.165 | 0.099 | |
| Mistral-3.2-24B-InstructApproach=Verbalized Top-k2025.11 | 78 | 70.5 | 0.158 | 0.047 | |
| Qwen3-4B-InstructApproach=Standard2025.11 | 76.4 | — | — | — | |
| Qwen3-4B-InstructApproach=Logit2025.11 | 76.4 | 67.5 | 0.223 | 0.219 | |
| Qwen3-4B-InstructApproach=p(True)2025.11 | 76.4 | 64.2 | 0.215 | 0.199 | |
| Qwen3-4B-InstructApproach=Verbalized Distrib.2025.11 | 75.6 | 67.2 | 0.173 | 0.044 | |
| Qwen3-4B-InstructApproach=Verbalized Top-k2025.11 | 74.9 | 67.5 | 0.207 | 0.181 | |
| Qwen3-4B-InstructApproach=Verbalized Conf.2025.11 | 73.5 | 66.3 | 0.238 | 0.224 |