Constraint Satisfaction Plan Generation on TravelPlanner
100Delivery RateReflexion
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ReflexionLLM=Gemini-3.1-flash-lite2026.05 | 100 | 80.5 | 26.7 | 77.9 | 55.6 | 12.8 | |
| MIRRORLLM=GPT-4o2026.05 | 100 | 73.2 | 13.9 | 27.6 | 13.3 | 2.2 | |
| EvoAgentLLM=Gemini-2.5-Pro2026.05 | 100 | 78.1 | 23.9 | 57.9 | 40.6 | 12.2 | |
| PMCLLM=Gemini-2.5-Pro2026.05 | 100 | 78.7 | 30.6 | 43.3 | 37.2 | 23.3 | |
| ATLASLLM=Gemini-2.5-Pro2026.05 | 100 | 88.5 | 48.3 | 82.6 | 74.4 | 44.4 | |
| SDPLLM=Gemini-3.1-flash-lite2026.05 | 100 | 95.2 | 61.7 | 97.4 | 93.8 | 61.7 | |
| SDPLLM=GPT-4o2026.05 | 100 | 95.6 | 65.6 | 97.4 | 93.9 | 65.6 | |
| Plan-and-SolveLLM=Gemini-3.1-flash-lite2026.05 | 99.4 | 84.7 | 26.1 | 78.9 | 56.7 | 13.3 | |
| GoalActLLM=Gemini-3.1-flash-lite2026.05 | 99.4 | 83.1 | 25 | 78.7 | 56.1 | 15 | |
| TDPLLM=Gemini-3.1-flash-lite2026.05 | 99.4 | 81.6 | 27.2 | 79.2 | 61.1 | 13.3 | |
| ReActLLM=Gemini-3.1-flash-lite2026.05 | 95 | 78.6 | 16.1 | 83.8 | 61.7 | 7.2 |