Medical Question Answering on MedCalc-Bench
35.3AccuracyZero-Shot CoT
Evaluation Results
| Method | Links | |
|---|---|---|
| Zero-Shot CoTModel=Claude 3.7 Sonnet2025.11 | 35.3 | |
| BFRSModel=o3 Mini2025.11 | 34.7 | |
| MIPROv2Model=Claude 3.7 Sonnet2025.11 | 34.7 | |
| MIPROv2Model=o3 Mini2025.11 | 34.3 | |
| Zero-Shot CoTModel=o3 Mini2025.11 | 34.2 | |
| BFRSModel=Claude 3.7 Sonnet2025.11 | 34.1 | |
| HELM BaselineModel=o3 Mini2025.11 | 34 | |
| Zero-Shot PredictModel=o3 Mini2025.11 | 33.4 | |
| BFRSModel=GPT 4o2025.11 | 27 | |
| MIPROv2Model=GPT 4o2025.11 | 26.8 | |
| Zero-Shot CoTModel=GPT 4o2025.11 | 26.6 | |
| Zero-Shot CoTModel=Gemini 2.0 Flash2025.11 | 26.3 | |
| MIPROv2Model=Gemini 2.0 Flash2025.11 | 25.4 | |
| BFRSModel=Gemini 2.0 Flash2025.11 | 25.2 | |
| BFRSModel=Qwen3 4B2025.11 | 22.7 | |
| Zero-Shot CoTModel=Llama 3.3 70B2025.11 | 22.5 | |
| MIPROv2Model=Qwen3 4B2025.11 | 21.5 | |
| HELM BaselineModel=Claude 3.7 Sonnet2025.11 | 21 | |
| MIPROv2Model=Llama 3.3 70B2025.11 | 21 | |
| Zero-Shot CoTModel=Qwen3 4B2025.11 | 20.8 | |
| Zero-Shot PredictModel=Claude 3.7 Sonnet2025.11 | 20.6 | |
| BFRSModel=Llama 3.3 70B2025.11 | 20 | |
| HELM BaselineModel=GPT 4o2025.11 | 18.8 | |
| Zero-Shot PredictModel=Gemini 2.0 Flash2025.11 | 17 | |
| HELM BaselineModel=Gemini 2.0 Flash2025.11 | 15.8 | |
| Zero-Shot PredictModel=GPT 4o2025.11 | 15.7 | |
| Zero-Shot PredictModel=Qwen3 4B2025.11 | 11.5 | |
| HELM BaselineModel=Llama 3.3 70B2025.11 | 11.3 | |
| Zero-Shot PredictModel=Llama 3.3 70B2025.11 | 9.9 | |
| HELM BaselineModel=Qwen3 4B2025.11 | 4.7 |