Taxi Domain on Taxi Standard
83.98AccuracyOpenAI o4-mini
Evaluation Results
| Method | Links | |
|---|---|---|
| OpenAI o4-minicategory=Proprietary Models2025.11 | 83.98 | |
| OpenAI o3category=Proprietary Models2025.11 | 72.66 | |
| Claude 4.5 Sonnetcategory=Proprietary Models2025.11 | 66.41 | |
| WMAct2025.11 | 62.16 | |
| PPO - Interactivemode=interactive2025.11 | 39.16 | |
| PPO - EntirePlanmode=single-turn output2025.11 | 38.92 | |
| GPT-5category=Proprietary Models2025.11 | 36.72 | |
| Gemini 2.5 Procategory=Proprietary Models2025.11 | 17.97 | |
| Qwen3-8Bcategory=Opensource Models2025.11 | 9.38 | |
| Qwen3-14Bcategory=Opensource Models2025.11 | 6.64 | |
| GPT-4ocategory=Proprietary Models2025.11 | 6.25 | |
| Qwen3-8B-Ownbackbone=Qwen3-8B2025.11 | 5.6 | |
| Qwen2.5-32B-Instructcategory=Opensource Models2025.11 | 2.34 | |
| Qwen2.5-7B-Instructcategory=Opensource Models2025.11 | 1.17 |