# Medical Calculation and Tool Use on MedMCP-Calc Overall

Source: [Medical Calculation and Tool Use on MedMCP-Calc Overall](<https://www.sota2.com/research/sota/medical-calculation-and-tool-use-on-medmcp-calc-overall>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T06:06:07.704Z

Evaluation of model performance on the overall MedMCP-Calc dataset for medical calculation and tool use tasks, covering all included sub-domains.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Medical Calculation and Tool Use |
| Dataset (including any stated split) | MedMCP-Calc Overall |
| Metric | Task Success Rate |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 25 of 25 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Task Success Rate](<https://www.sota2.com/research/sota/medical-calculation-and-tool-use-on-medmcp-calc-overall.md?metric=Task%20Success%20Rate>): Higher is better; unit: PERCENT.

- [Calculation Score](<https://www.sota2.com/research/sota/medical-calculation-and-tool-use-on-medmcp-calc-overall.md?metric=Calculation%20Score>): Higher is better; unit: PERCENT.

- [Execution Accuracy](<https://www.sota2.com/research/sota/medical-calculation-and-tool-use-on-medmcp-calc-overall.md?metric=Execution%20Accuracy>): Higher is better; unit: PERCENT.

- [Quality Performance Score](<https://www.sota2.com/research/sota/medical-calculation-and-tool-use-on-medmcp-calc-overall.md?metric=Quality%20Performance%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Task Success Rate | Result date | Report paper |
| --- | --- | --- | --- |
| Claude Opus 4.5 | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Claude Sonnet 4.5 | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| GPT-5 | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Gemini-2.5-Pro | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-235B-Instruct-2507 | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-235B-Thinking-2507 | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-235B non-thinking | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-235B thinking | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Llama-3.3-70B | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-30B-Instruct-2507 | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-30B-Thinking-2507 | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-4B-Instruct-2507 | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| CalcMate-4B | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| CalcMate-30B | 100 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Deepseek-V3.1 | 99.15 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-VL-235B | 99.15 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-Next-80B-Instruct | 99.15 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-Next-80B-Thinking | 99.15 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Llama-3.1-70B | 99.15 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Gemini-3-Pro | 97.46 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen2.5-72B | 97.46 | Not recorded | 69802fd0a8b01e76cc8e17af |
| MedGemma-27B-IT | 97.46 | Not recorded | 69802fd0a8b01e76cc8e17af |
| GLM-4.7 | 96.61 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Kimi-K2-Instruct | 96.61 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Baichuan-M2-32B | 79.66 | Not recorded | 69802fd0a8b01e76cc8e17af |

### Claude Opus 4.5

Task Success Rate: 100. Result ID: 699614445698665ec453bcee.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### Claude Sonnet 4.5

Task Success Rate: 100. Result ID: 699614445698665ec453bcef.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### GPT-5

Task Success Rate: 100. Result ID: 699614445698665ec453bcf0.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### Gemini-2.5-Pro

Task Success Rate: 100. Result ID: 699614445698665ec453bcf2.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c311e209131cea8cef5.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### Qwen3-235B-Instruct-2507

Task Success Rate: 100. Result ID: 699614445698665ec453bcf5.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235 B"}. This record does not assert independent reproduction.

### Qwen3-235B-Thinking-2507

Task Success Rate: 100. Result ID: 699614445698665ec453bcf6.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235 B"}. This record does not assert independent reproduction.

### Qwen3-235B non-thinking

Task Success Rate: 100. Result ID: 699614445698665ec453bcf8.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235 B"}. This record does not assert independent reproduction.

### Qwen3-235B thinking

Task Success Rate: 100. Result ID: 699614445698665ec453bcf9.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235 B"}. This record does not assert independent reproduction.

### Llama-3.3-70B

Task Success Rate: 100. Result ID: 699614445698665ec453bcfd.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Parameters":"70 B"}. This record does not assert independent reproduction.

### Qwen3-30B-Instruct-2507

Task Success Rate: 100. Result ID: 699614445698665ec453bd00.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"30 B"}. This record does not assert independent reproduction.

### Qwen3-30B-Thinking-2507

Task Success Rate: 100. Result ID: 699614445698665ec453bd01.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"30 B"}. This record does not assert independent reproduction.

### Qwen3-4B-Instruct-2507

Task Success Rate: 100. Result ID: 699614445698665ec453bd02.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"4 B"}. This record does not assert independent reproduction.

### CalcMate-4B

Task Success Rate: 100. Result ID: 699614445698665ec453bd05.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"4 B"}. This record does not assert independent reproduction.

### CalcMate-30B

Task Success Rate: 100. Result ID: 699614445698665ec453bd06.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"30 B"}. This record does not assert independent reproduction.

### Deepseek-V3.1

Task Success Rate: 99.15. Result ID: 699614445698665ec453bcf4.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77001e209131cea7066a.md>)

Recorded evaluation configuration: {"Parameters":"671 B"}. This record does not assert independent reproduction.

### Qwen3-VL-235B

Task Success Rate: 99.15. Result ID: 699614445698665ec453bcfa.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235B"}. This record does not assert independent reproduction.

### Qwen3-Next-80B-Instruct

Task Success Rate: 99.15. Result ID: 699614445698665ec453bcfb.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"80 B"}. This record does not assert independent reproduction.

### Qwen3-Next-80B-Thinking

Task Success Rate: 99.15. Result ID: 699614445698665ec453bcfc.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"80 B"}. This record does not assert independent reproduction.

### Llama-3.1-70B

Task Success Rate: 99.15. Result ID: 699614445698665ec453bcfe.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Parameters":"70 B"}. This record does not assert independent reproduction.

### Gemini-3-Pro

Task Success Rate: 97.46. Result ID: 699614445698665ec453bcf1.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### Qwen2.5-72B

Task Success Rate: 97.46. Result ID: 699614445698665ec453bcff.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"72 B"}. This record does not assert independent reproduction.

### MedGemma-27B-IT

Task Success Rate: 97.46. Result ID: 699614445698665ec453bd04.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79ff1e209131cea808a4.md>)

Recorded evaluation configuration: {"Parameters":"27 B"}. This record does not assert independent reproduction.

### GLM-4.7

Task Success Rate: 96.61. Result ID: 699614445698665ec453bcf3.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a891e209131cea83c41.md>)

Recorded evaluation configuration: {"Parameters":"355 B"}. This record does not assert independent reproduction.

### Kimi-K2-Instruct

Task Success Rate: 96.61. Result ID: 699614445698665ec453bcf7.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a4a1e209131cea823a1.md>)

Recorded evaluation configuration: {"Parameters":"1 T"}. This record does not assert independent reproduction.

### Baichuan-M2-32B

Task Success Rate: 79.66. Result ID: 699614445698665ec453bd03.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b041e209131cea863f2.md>)

Recorded evaluation configuration: {"Parameters":"32 B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
