Medical Question Answering on HeadQA
92.2AccuracyZero-Shot CoT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Zero-Shot CoTModel=Claude 3.7 Sonnet2025.11 | 92.2 | — | — | |
| MIPROv2Model=Claude 3.7 Sonnet2025.11 | 92.2 | — | — | |
| BFRSModel=Claude 3.7 Sonnet2025.11 | 92 | — | — | |
| HELM BaselineModel=Claude 3.7 Sonnet2025.11 | 91.2 | — | — | |
| BFRSModel=GPT 4o2025.11 | 91.1 | — | — | |
| MIPROv2Model=GPT 4o2025.11 | 91.1 | — | — | |
| Zero-Shot PredictModel=o3 Mini2025.11 | 90.9 | — | — | |
| Zero-Shot CoTModel=GPT 4o2025.11 | 90.7 | — | — | |
| HELM BaselineModel=GPT 4o2025.11 | 90.6 | — | — | |
| BFRSModel=o3 Mini2025.11 | 90.1 | — | — | |
| Zero-Shot CoTModel=o3 Mini2025.11 | 90 | — | — | |
| MIPROv2Model=Gemini 2.0 Flash2025.11 | 89.5 | — | — | |
| MIPROv2Model=o3 Mini2025.11 | 89.5 | — | — | |
| HELM BaselineModel=o3 Mini2025.11 | 89.3 | — | — | |
| Zero-Shot CoTModel=Gemini 2.0 Flash2025.11 | 89.3 | — | — | |
| BFRSModel=Gemini 2.0 Flash2025.11 | 88.9 | — | — | |
| Zero-Shot PredictModel=Claude 3.7 Sonnet2025.11 | 88.7 | — | — | |
| Zero-Shot PredictModel=Gemini 2.0 Flash2025.11 | 88.5 | — | — | |
| HELM BaselineModel=Gemini 2.0 Flash2025.11 | 88 | — | — | |
| Zero-Shot PredictModel=GPT 4o2025.11 | 86.4 | — | — | |
| BFRSModel=Llama 3.3 70B2025.11 | 86.2 | — | — | |
| MIPROv2Model=Llama 3.3 70B2025.11 | 86.2 | — | — | |
| Peer-reviewd reasoningModels=(1) + (4) + (5)2026.06 | 86.07 | 81.56 | 81.3 | |
| Zero-Shot CoTModel=Llama 3.3 70B2025.11 | 85.9 | — | — | |
| Peer-reviewd reasoningModels=(3) + (5)2026.06 | 85.66 | 81.15 | 80.6 | |
| Peer-reviewd reasoningModels=(2) + (4) + (5)2026.06 | 85.66 | 81.15 | 80.6 | |
| Peer-reviewd reasoningModels=(1) + (2) + (4) + (5)2026.06 | 85.66 | 81.15 | 80.9 | |
| Peer-reviewd reasoningModels=(4) + (5)2026.06 | 85.66 | 81.15 | 81 | |
| Peer-reviewd reasoningModels=(1) + (3) + (4) + (5)2026.06 | 85.66 | 81.15 | 82 | |
| HELM BaselineModel=Llama 3.3 70B2025.11 | 85.4 | — | — | |
| Peer-reviewd reasoningModels=(2) + (3) + (4) + (5)2026.06 | 85.25 | 80.74 | 81.2 | |
| Peer-reviewd reasoningModels=(1) + (2) + (3) + (4) + (5)2026.06 | 85.25 | 80.74 | 81.4 | |
| Peer-reviewd reasoningModels=(1) + (3) + (5)2026.06 | 84.84 | 80.33 | 80.7 | |
| Peer-reviewd reasoningModels=(3) + (4) + (5)2026.06 | 84.84 | 80.33 | 81.6 | |
| COT-based group votingModels=(3) + (4) + (5)2026.06 | 84.43 | 79.92 | 78.8 | |
| COT-based group votingModels=(4) + (5)2026.06 | 84.02 | 79.1 | 76.7 | |
| Chain-of-thoughtModel=GPT-oss-20B2026.06 | 83.61 | 78.69 | 77.7 | |
| Peer-reviewd reasoningModels=(1) + (5)2026.06 | 83.61 | 78.69 | 79 | |
| Peer-reviewd reasoningModels=(1) + (2) + (5)2026.06 | 83.61 | 78.69 | 79.1 | |
| Peer-reviewd reasoningModels=(2) + (3) + (5)2026.06 | 83.61 | 78.69 | 79.5 | |
| Peer-reviewd reasoningModels=(1) + (2) + (3) + (5)2026.06 | 83.61 | 78.69 | 79.9 | |
| Peer-reviewd reasoningModels=(2) + (5)2026.06 | 83.2 | 78.28 | 78.3 | |
| COT-based group votingModels=(2) + (4) + (5)2026.06 | 82.79 | 77.87 | 77.7 | |
| BFRSModel=Qwen3 4B2025.11 | 82.5 | — | — | |
| MIPROv2Model=Qwen3 4B2025.11 | 82.5 | — | — | |
| Chain-of-thoughtModel=Phi-42026.06 | 81.97 | 77.05 | 76.2 | |
| COT-based group votingModels=(2) + (3) + (4) + (5)2026.06 | 81.97 | 77.05 | 76.5 | |
| Zero-Shot PredictModel=Llama 3.3 70B2025.11 | 81.7 | — | — | |
| COT-based group votingModels=(1) + (4) + (5)2026.06 | 81.56 | 76.64 | 76.4 | |
| COT-based group votingModels=(1) + (2) + (3) + (4) + (5)2026.06 | 81.15 | 76.23 | 75.4 | |
| COT-based group votingModels=(2) + (3) + (5)2026.06 | 81.15 | 76.23 | 75.9 | |
| COT-based group votingModels=(1) + (3) + (4) + (5)2026.06 | 81.15 | 76.23 | 76.1 | |
| Zero-Shot CoTModel=Qwen3 4B2025.11 | 80.9 | — | — | |
| COT-based group votingModels=(3) + (5)2026.06 | 80.33 | 75.41 | 74.6 | |
| COT-based group votingModels=(2) + (3) + (4)2026.06 | 79.92 | 74.59 | 74.3 | |
| COT-based group votingModels=(3) + (4)2026.06 | 79.51 | 74.18 | 73.2 | |
| COT-based group votingModels=(1) + (2) + (4) + (5)2026.06 | 79.51 | 74.18 | 73.7 | |
| COT-based group votingModels=(1) + (3) + (5)2026.06 | 78.69 | 73.36 | 74.4 | |
| COT-based group votingModels=(1) + (2) + (3) + (5)2026.06 | 77.87 | 72.54 | 72.2 | |
| COT-based group votingModels=(1) + (3) + (4)2026.06 | 77.87 | 72.54 | 72.8 | |
| Zero-Shot PredictModel=Qwen3 4B2025.11 | 76.9 | — | — | |
| HELM BaselineModel=Qwen3 4B2025.11 | 76.7 | — | — | |
| Chain-of-thoughtModel=Llama-3.1-8B2026.06 | 72.13 | 66.39 | 71.1 | |
| Chain-of-thoughtModel=Qwen2.5-7B2026.06 | 70.9 | 65.16 | 65 | |
| Chain-of-thoughtModel=Deepseek-llm-7b2026.06 | 43.03 | 36.89 | 43.7 |