Mathematical and Spatial Reasoning on MathSpatial-Bench
100C3MHuman
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Human2026.02 | 100 | 89.4 | 97.6 | 98.2 | 96.1 | 97 | 95.2 | 96.5 | 94.4 | 97.7 | 96.2 | 96.3 | |
| Gemini-2.5-flashAvg. Token=1115.22026.02 | 34.3 | 42.5 | 46.7 | 26.7 | 58.5 | 47.8 | 43.8 | 50 | 25 | 57.4 | 33.3 | 48.5 | |
| Gemini-2.5-proAvg. Token=913.82026.02 | 31.4 | 42.5 | 43.5 | 26.7 | 58.5 | 44.9 | 28.1 | 45.6 | 41.7 | 49.6 | 33.3 | 44.9 | |
| GPT-5Avg. Token=676.32026.02 | 28.6 | 53.7 | 66.2 | 48.2 | 61.6 | 44.8 | 71.7 | 52.3 | 60 | 68.1 | 57.7 | 58.5 | |
| Llama-4-Maverick-17B-128EAvg. Token=845.22026.02 | 24.5 | 32.5 | 41 | 25 | 29.1 | 4.5 | 47.6 | 4.6 | 15.2 | 29.6 | 30.8 | 23.8 | |
| MathSpatial-Llama3-8BAvg. Token=397.32026.02 | 16.3 | 30.1 | 24.5 | 41.1 | 20.4 | 30.6 | 27.6 | 10.2 | 20.8 | 16.4 | 7.7 | 20.3 | |
| Llama3-8BAvg. Token=785.42026.02 | 10.2 | 15.4 | 20.7 | 17.9 | 15.1 | 20.9 | 23.4 | 9.3 | 13.6 | 12.2 | 7.7 | 15 | |
| Claude 4Avg. Token=1005.52026.02 | 6.1 | 33.3 | 46.2 | 21.4 | 30.8 | 1.5 | 60.7 | 0.2 | 32 | 41.3 | 53.8 | 26.7 | |
| Pixtral-12BAvg. Token=692.32026.02 | 4.1 | 22.8 | 34.1 | 26.8 | 20.4 | 1.5 | 34.5 | 0 | 8 | 12.2 | 3.8 | 15.3 | |
| GLM-4.5VAvg. Token=1391.12026.02 | 4.1 | 32.5 | 43.4 | 30.4 | 23 | 1.5 | 57.9 | 0 | 22.4 | 14.1 | 30.8 | 21 | |
| MathSpatial-Qwen2.5-VL-7BAvg. Token=351.92026.02 | 4.1 | 40.7 | 46.2 | 30.4 | 27.7 | 0.7 | 46.2 | 0 | 21.6 | 17.4 | 26.9 | 22.1 | |
| GPT-4.1Avg. Token=676.32026.02 | 2 | 45.5 | 43.1 | 26.8 | 24.1 | 0 | 55.2 | 0 | 29.6 | 18.3 | 46.2 | 22.6 | |
| GPT-4oAvg. Token=677.42026.02 | 2 | 36.6 | 34.1 | 33.9 | 23.5 | 1.5 | 45.5 | 0 | 26.4 | 16.4 | 26.9 | 19.6 | |
| Claude 3.7Avg. Token=885.82026.02 | 2 | 36.6 | 39.3 | 21.4 | 21.6 | 1.5 | 57.9 | 0 | 23.2 | 26.3 | 34.6 | 21.5 | |
| Claude 3.5Avg. Token=858.62026.02 | 2 | 32.5 | 42.8 | 23.2 | 26.9 | 0.7 | 57.2 | 0 | 24.8 | 21.1 | 46.2 | 22.3 | |
| Kimi-VL-A3B-ThinkingAvg. Token=785.22026.02 | 2 | 33.3 | 31.7 | 50 | 23.8 | 0 | 47.6 | 0.6 | 23.2 | 5.6 | 11.5 | 18.2 | |
| Qwen2.5-VL-72B-InstructAvg. Token=497.62026.02 | 2 | 33.3 | 35.9 | 23.2 | 26.9 | 0 | 47.6 | 0 | 20.8 | 17.8 | 19.2 | 19.7 | |
| MathSpatial-InternVL3-8BAvg. Token=318.32026.02 | 2 | 50.4 | 36.2 | 33.9 | 22.7 | 11.9 | 56.6 | 3.9 | 23.2 | 15.5 | 15.4 | 22.6 | |
| Qwen2.5-VL-7BAvg. Token=465.32026.02 | 0 | 32.5 | 34.8 | 17.9 | 24.4 | 0 | 41.4 | 0 | 20 | 13.6 | 15.4 | 17.8 | |
| DeepSeek-VL2-7BAvg. Token=397.42026.02 | 0 | 14.6 | 11.7 | 7.1 | 5.9 | 4.5 | 2.1 | 2.1 | 11.2 | 4.2 | 7.7 | 6.6 | |
| InternVL3-8BAvg. Token=473.52026.02 | 0 | 29.3 | 35.9 | 23.2 | 21.3 | 0.7 | 40 | 0 | 19.2 | 14.6 | 15.4 | 17.4 | |
| Idefics3-8BAvg. Token=294.52026.02 | 0 | 22.8 | 25.9 | 14.3 | 18.2 | 1.5 | 22.1 | 0 | 8 | 6.1 | 0 | 11.7 |