Outcome correctness on TPS-CalcBench core set v2
51.9Exact Matchgpt-5.2
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| gpt-5.22026.04 | 51.9 | 29 | 9.8 | 9.3 | 87.6 | 95 | |
| gpt-5.12026.04 | 48.1 | 27.9 | 10 | 14 | 85.2 | 95.7 | |
| glm-52026.04 | 47.1 | 29.3 | 13.8 | 9.8 | 86.2 | 100 | |
| deepseek-v3.12026.04 | 43.1 | 28.1 | 10 | 18.8 | 81.7 | 87.6 | |
| aws.claude-opus-4.52026.04 | 41.9 | 24.3 | 14.5 | 19.3 | 85 | 100 | |
| Doubao-Seed-1.82026.04 | 38.6 | 28.1 | 14.5 | 18.8 | 75.7 | 100 | |
| glm-4.72026.04 | 37.6 | 24.3 | 13.6 | 24.5 | 76.4 | 100 | |
| aws.claude-sonnet-4.52026.04 | 33.6 | 28.3 | 14.8 | 23.3 | 72.1 | 100 | |
| aws.claude-haiku-4.52026.04 | 24.3 | 28.1 | 18.8 | 28.8 | 66.2 | 100 | |
| gemini-3-flash-preview2026.04 | 9.8 | 13.8 | 24.3 | 52.1 | 52.6 | 100 | |
| MiniMax-M2.52026.04 | 0 | 5 | 14.8 | 80.2 | 24.3 | 97.1 | |
| MiniMax-M2.12026.04 | 0 | 4.5 | 9.8 | 85.7 | 18.8 | 97.9 |