Trip Planning on NaturalPlan Trip 2024
90.7Final CSRPDP
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| PDPModel=Grok-3-mini2026.01 | 90.7 | 85.6 | 89.7 | |
| PDPModel=Grok-32026.01 | 90.2 | 76 | 78.1 | |
| LEAPModel=Grok-3-mini2026.01 | 90.2 | 84.7 | 85.4 | |
| PDP(+)Model=Grok-32026.01 | 89.8 | 76.6 | 78.9 | |
| LEAPModel=Grok-32026.01 | 87.9 | 76.8 | 79.1 | |
| PDP(+)Model=Grok-3-mini2026.01 | 85.7 | 85.4 | 87.6 | |
| PDPModel=Llama-3.3-70B2026.01 | 82 | 78.2 | 80.2 | |
| BaseModel=Grok-3-mini2026.01 | 81.5 | 76.8 | 84.7 | |
| PDP(+)Model=Llama-3.3-70B2026.01 | 81 | 76.6 | 79.1 | |
| PDP(+)Model=GPT-oss-120B2026.01 | 80.3 | 76.8 | 78.4 | |
| PDPModel=GPT-oss-120B2026.01 | 79.5 | 76.7 | 78.3 | |
| PDP(+)Model=GPT-4.12026.01 | 76.6 | 70.7 | 71.9 | |
| PDPModel=GPT-4.12026.01 | 76.3 | 70.2 | 71.4 | |
| LEAPModel=Llama-3.3-70B2026.01 | 76.2 | 74.6 | 78.4 | |
| BaseModel=Grok-32026.01 | 74.7 | 66.8 | 68.4 | |
| LEAPModel=GPT-4.12026.01 | 74.6 | 70.8 | 72.2 | |
| BaseModel=Llama-3.3-70B2026.01 | 71.5 | 73.6 | 74.1 | |
| BaseModel=GPT-4.12026.01 | 69.4 | 68.2 | 69.6 | |
| PDP(+)Model=GPT-4o2026.01 | 66.2 | 60.9 | 62.3 | |
| PDPModel=GPT-4o2026.01 | 62.8 | 58.7 | 60.2 | |
| PDP(+)Model=GPT-oss-20B2026.01 | 58.6 | 45.3 | 47.3 | |
| BaseModel=GPT-4o2026.01 | 55.7 | 55.1 | 57 | |
| PDPModel=GPT-oss-20B2026.01 | 55.6 | 47.5 | 50.8 | |
| BaseModel=GPT-oss-20B2026.01 | 51.5 | 46.4 | 47.7 | |
| LEAPModel=GPT-oss-120B2026.01 | 46.1 | 38.3 | 45.2 | |
| LEAPModel=GPT-4o2026.01 | 44.1 | 49 | 52.4 | |
| BaseModel=GPT-oss-120B2026.01 | 36.7 | 37.9 | 45.9 | |
| LEAPModel=GPT-oss-20B2026.01 | 36.1 | 33.6 | 37.8 |