Medical Calculation and Tool Use on MedMCP-Calc Internal Medicine & Organ Systems
69.63Calculation Success (CS)Claude Opus 4.5
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Claude Opus 4.5Parameters=/2026.01 | 69.63 | 73.89 | 37.35 | |
| Claude Sonnet 4.5Parameters=/2026.01 | 67.77 | 56.64 | 33.1 | |
| GPT-5Parameters=/2026.01 | 62.77 | 56.64 | 29.64 | |
| CalcMate-30BParameters=30 B2026.01 | 62.76 | 76.14 | 20.73 | |
| Gemini-2.5-ProParameters=/2026.01 | 58.89 | 41.35 | 21.76 | |
| Gemini-3-ProParameters=/2026.01 | 56.37 | 59.1 | 31.17 | |
| CalcMate-4BParameters=4 B2026.01 | 54 | 69.74 | 14.05 | |
| GLM-4.7Parameters=355 B2026.01 | 51.03 | 52.14 | 17.57 | |
| Deepseek-V3.1Parameters=671 B2026.01 | 45.35 | 39.7 | 14.22 | |
| Kimi-K2-InstructParameters=1 T2026.01 | 43.52 | 28.62 | 9.07 | |
| Qwen3-Next-80B-InstructParameters=80 B2026.01 | 42.62 | 24.57 | 9.99 | |
| Qwen3-235B-Instruct-2507Parameters=235 B2026.01 | 41.31 | 32.62 | 10.29 | |
| Qwen3-235B-Thinking-2507Parameters=235 B2026.01 | 39.81 | 27.7 | 12.9 | |
| Qwen3-VL-235BParameters=235B2026.01 | 37.16 | 27.31 | 11.76 | |
| Baichuan-M2-32BParameters=32 B2026.01 | 36.31 | 36.72 | 9.23 | |
| Qwen3-235B thinkingParameters=235 B2026.01 | 34.67 | 31.41 | 9.61 | |
| Qwen3-Next-80B-ThinkingParameters=80 B2026.01 | 34.09 | 35.97 | 13.33 | |
| Qwen2.5-72BParameters=72 B2026.01 | 33.21 | 38.81 | 8.45 | |
| MedGemma-27B-ITParameters=27 B2026.01 | 32.45 | 20.49 | 4.54 | |
| Qwen3-30B-Instruct-2507Parameters=30 B2026.01 | 32.41 | 24.74 | 5.94 | |
| Qwen3-235B non-thinkingParameters=235 B2026.01 | 32.4 | 38.5 | 7.22 | |
| Qwen3-30B-Thinking-2507Parameters=30 B2026.01 | 29.87 | 22.95 | 10.08 | |
| Qwen3-4B-Instruct-2507Parameters=4 B2026.01 | 19.3 | 8.09 | 1.45 | |
| Llama-3.1-70BParameters=70 B2026.01 | 12.55 | 23.24 | 1.19 | |
| Llama-3.3-70BParameters=70 B2026.01 | 11.57 | 23.49 | 2.61 |