Planning on TravelPlanner #180 (val)
95.64CS-MicroHiMAP-Travel
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| HiMAP-TravelBackbone=Qwen3-8B2026.03 | 95.64 | — | 73.61 | 76.85 | 63.33 | 52.78 | — | 100 | 100 | |
| HiMAP-TravelBackbone=Qwen3-4B2026.03 | 95.57 | — | 72.36 | 70.89 | 59.17 | 48.61 | — | 100 | 100 | |
| AutoRefineAugmentation=ReAct2026.01 | 94.24 | 100 | 57.78 | 52.86 | 43.89 | — | 40 | — | — | |
| AutoRefine2026.01 | 92.71 | 100 | 54.44 | 49.76 | 38.89 | — | 35.56 | — | — | |
| DeepTravelBackbone=Qwen3-8B2026.03 | 92.6 | — | 67.64 | 71.43 | 58.61 | 45.56 | — | 97.6 | 99.4 | |
| DeepTravelBackbone=Qwen3-4B2026.03 | 90 | — | 59.58 | 63.15 | 49.44 | 35.28 | — | 96.8 | 98.2 | |
| MTPBackbone=GPT-4 w/ hint2026.03 | 87.29 | — | 43.33 | 47.14 | 46.67 | 33.33 | — | 96.7 | 97.4 | |
| ATLASAugmentation=Hint2026.01 | 85.42 | 100 | 48.33 | 60.71 | 47.22 | — | 33.33 | — | — | |
| ATLAS2026.01 | 83.4 | 100 | 37.78 | 56.43 | 38.89 | — | 23.33 | — | — | |
| MegaAgentMode=sole-planning2024.08 | 81.88 | 100 | 27.22 | 40.48 | 23.89 | 10 | — | — | — | |
| Direct GPT-4-TurboBackbone=GPT-4-Turbo, Strategy=Direct, Mode=sole-planning2024.08 | 80.4 | 100 | 17.2 | 47.1 | 22.2 | 4.4 | — | — | — | |
| ReAct2026.01 | 79.38 | 100 | 22.78 | 56.19 | 38.89 | — | 11.67 | — | — | |
| PMC2026.01 | 76.11 | 96.67 | 21.67 | 39.52 | 30.56 | — | 14.44 | — | — | |
| ReActAugmentation=Reflexion2026.01 | 74.79 | 99.44 | 18.33 | 45.48 | 28.33 | — | 10 | — | — | |
| ReActAugmentation=EvoAgent2026.01 | 74.08 | 99.44 | 19.71 | 38.05 | 20.33 | — | 8.03 | — | — | |
| Direct Mixtral-8×7B-MoEBackbone=Mixtral-8×7B-MoE, Strategy=Direct, Mode=sole-planning2024.08 | 68.1 | 100 | 5 | 3.3 | 1.1 | 0 | — | — | — | |
| CoT GPT-3.5-TurboBackbone=GPT-3.5-Turbo, Strategy=Chain-of-Thought, Mode=sole-planning2024.08 | 66.3 | 100 | 3.3 | 11.9 | 5 | 0 | — | — | — | |
| Direct Gemini ProBackbone=Gemini Pro, Strategy=Direct, Mode=sole-planning2024.08 | 65 | 93.9 | 8.3 | 9.3 | 4.4 | 0.6 | — | — | — | |
| Direct GPT-3.5-TurboBackbone=GPT-3.5-Turbo, Strategy=Direct, Mode=sole-planning2024.08 | 60.2 | 100 | 4.4 | 11 | 2.8 | 0 | — | — | — | |
| Reflexion GPT-3.5-TurboBackbone=GPT-3.5-Turbo, Strategy=Reflexion, Mode=sole-planning2024.08 | 53.8 | 93.9 | 2.8 | 11 | 2.8 | 0 | — | — | — | |
| ATLASBackbone=Gemini-2.5-Pro2026.03 | 48.33 | — | 88.54 | 82.62 | 74.44 | 44.44 | — | 100 | 100 | |
| ReAct GPT-3.5-TurboBackbone=GPT-3.5-Turbo, Strategy=ReAct, Mode=sole-planning2024.08 | 47.6 | 82.2 | 3.9 | 11.4 | 6.7 | 0.6 | — | — | — |