Travel Planning on TravelPlanner
14.8Average Tokens UsedToRL
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ToRLBackbone=Qwen-2.5-7B-Instruct2025.10 | 14.8 | — | — | 86.7 | — | |
| AgentFlowBackbone=Qwen-2.5-7B-Instruct2025.10 | 16.2 | — | — | 91.7 | — | |
| SimpleTIRBackbone=Qwen-2.5-7B-Instruct2025.10 | 16.2 | — | — | 88.9 | — | |
| GRPOfmBackbone=Qwen-2.5-7B-Instruct2025.10 | 16.4 | — | — | 90.6 | — | |
| ARPOfmBackbone=Qwen-2.5-7B-Instruct2025.10 | 16.7 | — | — | 92.2 | — | |
| PORToolBackbone=Qwen-2.5-7B-Instruct2025.10 | 16.7 | — | — | 92.8 | — | |
| PLAN-AND-BUDGET + LinearModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,136.18 | 19.56 | 0.34 | — | — | |
| PLAN-AND-BUDGET + CosineModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,140.79 | 20.22 | 0.36 | — | — | |
| PLAN-AND-BUDGET + PolynomialModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,148.14 | 32.67 | 0.93 | — | — | |
| PLAN-AND-BUDGET + PolynomialModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,148.53 | 23.11 | 0.47 | — | — | |
| PLAN-AND-BUDGET + ExponentialModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,156.64 | 21.44 | 0.4 | — | — | |
| Direct Global BudgetModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,158.81 | 13.78 | 0.16 | — | — | |
| PLAN-AND-BUDGET + LinearModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,162.24 | 31.67 | 0.86 | — | — | |
| PLAN-AND-BUDGET + CosineModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,173.96 | 31.67 | 0.85 | — | — | |
| PLAN-AND-BUDGET + ExponentialModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,187.85 | 32 | 0.86 | — | — | |
| PLAN-AND-BUDGET + WeightedModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,197.32 | 29.67 | 0.74 | — | — | |
| Direct Global BudgetModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,215.29 | 24.33 | 0.49 | — | — | |
| Planned Global BudgetModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,220.41 | 30.67 | 0.77 | — | — | |
| PLAN-AND-BUDGET + WeightedModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,222.09 | 23.33 | 0.45 | — | — | |
| PLAN-AND-BUDGET + UniformModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,227.99 | 20.67 | 0.35 | — | — | |
| PLAN-AND-BUDGET + UniformModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,232.98 | 31.56 | 0.81 | — | — | |
| Planned Global BudgetModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,241.19 | 22.56 | 0.41 | — | — | |
| Direct Global BudgetModel=o4-mini2025.05 | 1,248.53 | 8.33 | 0.056 | — | — | |
| PLAN-AND-BUDGET + CosineModel=o4-mini2025.05 | 1,252.06 | 9.79 | 0.077 | — | — | |
| PLAN-AND-BUDGET + PolynomialModel=o4-mini2025.05 | 1,266.11 | 11.49 | 0.104 | — | — | |
| PLAN-AND-BUDGET + LinearModel=o4-mini2025.05 | 1,306.54 | 11.66 | 0.103 | — | — | |
| PLAN-AND-BUDGET + ExponentialModel=o4-mini2025.05 | 1,307.87 | 9.91 | 0.075 | — | — | |
| Planned VanillaModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,343.67 | 20.22 | 0.3 | — | — | |
| PLAN-AND-BUDGET + UniformModel=o4-mini2025.05 | 1,345.32 | 11 | 0.09 | — | — | |
| PLAN-AND-BUDGET + WeightedModel=o4-mini2025.05 | 1,353.67 | 10.91 | 0.088 | — | — | |
| Direct VanillaModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,361.37 | 26.22 | 0.5 | — | — | |
| Planned Global BudgetModel=o4-mini2025.05 | 1,392.11 | 7.19 | 0.037 | — | — | |
| Direct VanillaModel=DeepSeek-R1-Distill-Qwen-32B2025.05 | 1,430.14 | 14.33 | 0.14 | — | — | |
| Planned VanillaModel=DeepSeek-R1-Distill-Llama-70B2025.05 | 1,464.5 | 30.67 | 0.64 | — | — | |
| Direct VanillaModel=o4-mini2025.05 | 1,559.65 | 11.58 | 0.086 | — | — | |
| Planned VanillaModel=o4-mini2025.05 | 1,640.46 | 12.2 | 0.091 | — | — | |
| PLAN-AND-BUDGET + CosineModel=QwQ-32B2025.05 | 2,496.46 | 36.18 | 0.52 | — | — | |
| PLAN-AND-BUDGET + PolynomialModel=QwQ-32B2025.05 | 2,511.35 | 35 | 0.49 | — | — | |
| Direct Global BudgetModel=QwQ-32B2025.05 | 2,530.04 | 30.78 | 0.37 | — | — | |
| PLAN-AND-BUDGET + LinearModel=QwQ-32B2025.05 | 2,671.7 | 34.55 | 0.45 | — | — | |
| PLAN-AND-BUDGET + ExponentialModel=QwQ-32B2025.05 | 2,724.23 | 35.44 | 0.46 | — | — | |
| PLAN-AND-BUDGET + WeightedModel=QwQ-32B2025.05 | 2,842.74 | 33.89 | 0.4 | — | — | |
| PLAN-AND-BUDGET + UniformModel=QwQ-32B2025.05 | 2,854.24 | 36 | 0.45 | — | — | |
| Planned Global BudgetModel=QwQ-32B2025.05 | 3,199.58 | 35.22 | 0.39 | — | — | |
| Direct VanillaModel=QwQ-32B2025.05 | 3,432.33 | 34.89 | 0.35 | — | — | |
| Planned VanillaModel=QwQ-32B2025.05 | 3,669.88 | 37.22 | 0.38 | — | — |