Optimal Route Generation on TransitLM 1,000 (test)
98.3ConnectivityGemini-3.1
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemini-3.1Augmentation=Tool-augmented, Tool=Amap transit routing API2026.05 | 98.3 | 99.5 | 98.7 | 82.7 | 82.5 | 73.3 | |
| Deepseek-V4Augmentation=Tool-augmented, Tool=Amap transit routing API2026.05 | 98 | 100 | 98.5 | 84.8 | 83.4 | 74.2 | |
| DoubaoAugmentation=Tool-augmented, Tool=Amap transit routing API2026.05 | 97.8 | 100 | 99.2 | 83.4 | 83.1 | 74.4 | |
| GPT-5.4Augmentation=Tool-augmented, Tool=Amap transit routing API2026.05 | 97.3 | 97.2 | 95.1 | 82 | 81 | 71.7 | |
| Qwen-3.6Augmentation=Tool-augmented, Tool=Amap transit routing API2026.05 | 96.7 | 98.7 | 97.9 | 81.3 | 81.3 | 72 | |
| Claude-4.6Augmentation=Tool-augmented, Tool=Amap transit routing API2026.05 | 96.5 | 99.5 | 97.4 | 82.3 | 81.2 | 72.1 |