# Medical Calculation and Tool Use on MedMCP-Calc Internal Medicine & Organ Systems

Source: [Medical Calculation and Tool Use on MedMCP-Calc Internal Medicine & Organ Systems](<https://www.sota2.com/research/sota/medical-calculation-and-tool-use-on-medmcp-calc-internal-medicine-and-organ-systems>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:38:54.648Z

Evaluation of model performance on the MedMCP-Calc dataset, focusing on Internal Medicine & Organ Systems, specifically testing Medical Calculation and Tool Use capabilities.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Medical Calculation and Tool Use |
| Dataset (including any stated split) | MedMCP-Calc Internal Medicine & Organ Systems |
| Metric | Calculation Success (CS) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 25 of 25 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Calculation Success (CS)](<https://www.sota2.com/research/sota/medical-calculation-and-tool-use-on-medmcp-calc-internal-medicine-and-organ-systems.md?metric=Calculation%20Success%20(CS)>): Higher is better; unit: PERCENT.

- [Execution Accuracy (EA)](<https://www.sota2.com/research/sota/medical-calculation-and-tool-use-on-medmcp-calc-internal-medicine-and-organ-systems.md?metric=Execution%20Accuracy%20(EA)>): Higher is better; unit: PERCENT.

- [Quality Percentage (QP)](<https://www.sota2.com/research/sota/medical-calculation-and-tool-use-on-medmcp-calc-internal-medicine-and-organ-systems.md?metric=Quality%20Percentage%20(QP)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Calculation Success (CS) | Result date | Report paper |
| --- | --- | --- | --- |
| Claude Opus 4.5 | 69.63 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Claude Sonnet 4.5 | 67.77 | Not recorded | 69802fd0a8b01e76cc8e17af |
| GPT-5 | 62.77 | Not recorded | 69802fd0a8b01e76cc8e17af |
| CalcMate-30B | 62.76 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Gemini-2.5-Pro | 58.89 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Gemini-3-Pro | 56.37 | Not recorded | 69802fd0a8b01e76cc8e17af |
| CalcMate-4B | 54 | Not recorded | 69802fd0a8b01e76cc8e17af |
| GLM-4.7 | 51.03 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Deepseek-V3.1 | 45.35 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Kimi-K2-Instruct | 43.52 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-Next-80B-Instruct | 42.62 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-235B-Instruct-2507 | 41.31 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-235B-Thinking-2507 | 39.81 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-VL-235B | 37.16 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Baichuan-M2-32B | 36.31 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-235B thinking | 34.67 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-Next-80B-Thinking | 34.09 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen2.5-72B | 33.21 | Not recorded | 69802fd0a8b01e76cc8e17af |
| MedGemma-27B-IT | 32.45 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-30B-Instruct-2507 | 32.41 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-235B non-thinking | 32.4 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-30B-Thinking-2507 | 29.87 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Qwen3-4B-Instruct-2507 | 19.3 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Llama-3.1-70B | 12.55 | Not recorded | 69802fd0a8b01e76cc8e17af |
| Llama-3.3-70B | 11.57 | Not recorded | 69802fd0a8b01e76cc8e17af |

### Claude Opus 4.5

Calculation Success (CS): 69.63. Result ID: 699614445698665ec453bca3.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### Claude Sonnet 4.5

Calculation Success (CS): 67.77. Result ID: 699614445698665ec453bca4.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### GPT-5

Calculation Success (CS): 62.77. Result ID: 699614445698665ec453bca5.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### CalcMate-30B

Calculation Success (CS): 62.76. Result ID: 699614445698665ec453bcbb.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"30 B"}. This record does not assert independent reproduction.

### Gemini-2.5-Pro

Calculation Success (CS): 58.89. Result ID: 699614445698665ec453bca7.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c311e209131cea8cef5.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### Gemini-3-Pro

Calculation Success (CS): 56.37. Result ID: 699614445698665ec453bca6.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"/"}. This record does not assert independent reproduction.

### CalcMate-4B

Calculation Success (CS): 54. Result ID: 699614445698665ec453bcba.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"4 B"}. This record does not assert independent reproduction.

### GLM-4.7

Calculation Success (CS): 51.03. Result ID: 699614445698665ec453bca8.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a891e209131cea83c41.md>)

Recorded evaluation configuration: {"Parameters":"355 B"}. This record does not assert independent reproduction.

### Deepseek-V3.1

Calculation Success (CS): 45.35. Result ID: 699614445698665ec453bca9.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77001e209131cea7066a.md>)

Recorded evaluation configuration: {"Parameters":"671 B"}. This record does not assert independent reproduction.

### Kimi-K2-Instruct

Calculation Success (CS): 43.52. Result ID: 699614445698665ec453bcac.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a4a1e209131cea823a1.md>)

Recorded evaluation configuration: {"Parameters":"1 T"}. This record does not assert independent reproduction.

### Qwen3-Next-80B-Instruct

Calculation Success (CS): 42.62. Result ID: 699614445698665ec453bcb0.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"80 B"}. This record does not assert independent reproduction.

### Qwen3-235B-Instruct-2507

Calculation Success (CS): 41.31. Result ID: 699614445698665ec453bcaa.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235 B"}. This record does not assert independent reproduction.

### Qwen3-235B-Thinking-2507

Calculation Success (CS): 39.81. Result ID: 699614445698665ec453bcab.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235 B"}. This record does not assert independent reproduction.

### Qwen3-VL-235B

Calculation Success (CS): 37.16. Result ID: 699614445698665ec453bcaf.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235B"}. This record does not assert independent reproduction.

### Baichuan-M2-32B

Calculation Success (CS): 36.31. Result ID: 699614445698665ec453bcb8.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b041e209131cea863f2.md>)

Recorded evaluation configuration: {"Parameters":"32 B"}. This record does not assert independent reproduction.

### Qwen3-235B thinking

Calculation Success (CS): 34.67. Result ID: 699614445698665ec453bcae.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235 B"}. This record does not assert independent reproduction.

### Qwen3-Next-80B-Thinking

Calculation Success (CS): 34.09. Result ID: 699614445698665ec453bcb1.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"80 B"}. This record does not assert independent reproduction.

### Qwen2.5-72B

Calculation Success (CS): 33.21. Result ID: 699614445698665ec453bcb4.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

Recorded evaluation configuration: {"Parameters":"72 B"}. This record does not assert independent reproduction.

### MedGemma-27B-IT

Calculation Success (CS): 32.45. Result ID: 699614445698665ec453bcb9.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79ff1e209131cea808a4.md>)

Recorded evaluation configuration: {"Parameters":"27 B"}. This record does not assert independent reproduction.

### Qwen3-30B-Instruct-2507

Calculation Success (CS): 32.41. Result ID: 699614445698665ec453bcb5.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"30 B"}. This record does not assert independent reproduction.

### Qwen3-235B non-thinking

Calculation Success (CS): 32.4. Result ID: 699614445698665ec453bcad.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"235 B"}. This record does not assert independent reproduction.

### Qwen3-30B-Thinking-2507

Calculation Success (CS): 29.87. Result ID: 699614445698665ec453bcb6.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"30 B"}. This record does not assert independent reproduction.

### Qwen3-4B-Instruct-2507

Calculation Success (CS): 19.3. Result ID: 699614445698665ec453bcb7.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Parameters":"4 B"}. This record does not assert independent reproduction.

### Llama-3.1-70B

Calculation Success (CS): 12.55. Result ID: 699614445698665ec453bcb3.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Parameters":"70 B"}. This record does not assert independent reproduction.

### Llama-3.3-70B

Calculation Success (CS): 11.57. Result ID: 699614445698665ec453bcb2.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd0a8b01e76cc8e17af.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Parameters":"70 B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
