Agent Tool-Use Planning on ThinkGeo 14 Tools (test)
77.7Tool Selection Accuracy (TSA)Qwen3-235B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen3-235BApproach=Meta2026.05 | 77.7 | 74.3 | 55 | 30.7 | 88.6 | 76.7 | 62.5 | |
| Qwen3-8BApproach=Meta2026.05 | 74.3 | 74.1 | 51.9 | 24.6 | 76.9 | 65.9 | 50.6 | |
| Qwen2.5-72BApproach=Meta2026.05 | 72 | 72 | 49.7 | 21.9 | 72.7 | 62.5 | 39.5 | |
| GPT-5Approach=Meta2026.05 | 52.9 | 52.8 | 25.9 | 40 | 84.1 | 73.1 | 50.9 | |
| Qwen2.5-72BApproach=Base2026.05 | 51.1 | 51.1 | 34.4 | 26.3 | 71.8 | 59.2 | 8 | |
| Qwen3-8BApproach=Base2026.05 | 50.2 | 50.1 | 34.8 | 19.9 | 76.9 | 60.6 | 14.8 | |
| GPT-5Approach=Base2026.05 | 48.4 | 48.4 | 29.3 | 53.5 | 70.9 | 57.1 | 39.1 | |
| Qwen3-235BApproach=Base2026.05 | 48.3 | 45.5 | 39.2 | 29 | 84.5 | 69.1 | 34.8 |