Medical Calculation on MedCalc-Bench Original (test)
73.95AccuracyDeepSeek-R1 (MedCalc-R1)
Evaluation results
| Method | Links | |
|---|---|---|
| DeepSeek-R1 (MedCalc-R1)Approach=RL + verifiable rewards, Dataset=Orig (train/test 9,765/1,048), Setting=SFT + GRPO2026.02 | 73.95 | |
| Qwen3-8BApproach=RL on recomputed labels, Dataset=Orig (recomputed; test n=1,047), Setting=GRPO2026.02 | 71.4 | |
| o1-miniApproach=RL + verifiable rewards, Dataset=Orig (train/test 9,765/1,048), Setting=SFT + GRPO2026.02 | 67.84 | |
| RiskAgent-GPT-4oApproach=Agentic tool framework, Dataset=Orig (external eval), Setting=Zero-shot CoT + tools2026.02 | 67.71 | |
| MedCalc-R1 (3B)Approach=RL + verifiable rewards, Dataset=Orig (train/test 9,765/1,048), Setting=SFT + GRPO2026.02 | 51.34 | |
| GPT-4Approach=Prompting baseline, Dataset=Orig (test 1,047), Setting=One-shot CoT2026.02 | 50.9 | |
| GPT-4Approach=Code-exec prompting, Dataset=Orig, Setting=Zero-shot CoT + code2026.02 | 48.51 | |
| GPT-3.5Approach=Code-exec prompting, Dataset=Orig, Setting=Zero-shot CoT + code2026.02 | 30.29 |