Trip Planning on Natural Plan Trip Planning
33.2Success RateSMaRT
Evaluation Results
| Method | Links | |
|---|---|---|
| SMaRTModel=Gemini-1.5, Number of shots=5-shot2025.10 | 33.2 | |
| DirectModel=Gemini-1.5, Number of shots=5-shot2025.10 | 32.2 | |
| CoTModel=Gemini-1.5, Number of shots=5-shot2025.10 | 31.2 | |
| LLM-as-a-JudgeModel=Gemini-1.5, Number of shots=5-shot2025.10 | 29.2 | |
| SMaRTModel=GPT-4, Number of shots=5-shot2025.10 | 24.5 | |
| CoTModel=GPT-4, Number of shots=5-shot2025.10 | 9.4 | |
| LLM-as-a-JudgeModel=GPT-4, Number of shots=5-shot2025.10 | 6.2 | |
| DirectModel=GPT-4, Number of shots=5-shot2025.10 | 5.6 |