Task Planning on RestBench TMDB
82.63Node F1SGC
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SGCLLM=GPT-3.5-turbo2024.05 | 82.63 | 59.15 | 1.61 | |
| DirectLLM=GPT-4-turbo2024.05 | 82.56 | 56.67 | 1.75 | |
| SGCLLM=GPT-4-turbo2024.05 | 82.27 | 56.37 | 1.62 | |
| AdaptiveSearchLLM=GPT-4-turbo2024.05 | 81.78 | 55.15 | 6.35 | |
| DirectLLM=GPT-3.5-turbo2024.05 | 81.7 | 57.52 | 1.67 | |
| BeamSearchLLM=GPT-3.5-turbo2024.05 | 81.24 | 57.98 | 6.42 | |
| BeamSearchLLM=GPT-4-turbo2024.05 | 80.11 | 51 | 5.18 | |
| AdaptiveSearchLLM=GPT-3.5-turbo2024.05 | 77.57 | 53.65 | 5.89 | |
| Greedy SearchLLM=GPT-3.5-turbo2024.05 | 76.19 | 50.11 | 3.06 | |
| Greedy SearchLLM=GPT-4-turbo2024.05 | 75.34 | 49.95 | 3.73 | |
| SGCLLM=CodeLlama 13B2024.05 | 71.4 | 47.55 | 1.9 | |
| DirectLLM=CodeLlama 13B2024.05 | 68.91 | 43.74 | 2.02 | |
| BeamSearchLLM=CodeLlama 13B2024.05 | 68.08 | 42.92 | 8.88 | |
| Greedy SearchLLM=CodeLlama 13B2024.05 | 66.67 | 42.16 | 3.81 | |
| SGCLLM=Mistral 7B2024.05 | 64.72 | 25.67 | 1.89 | |
| AdaptiveSearchLLM=CodeLlama 13B2024.05 | 63.74 | 37.17 | 8.16 | |
| BeamSearchLLM=Mistral 7B2024.05 | 63.41 | 26.79 | 11.26 | |
| DirectLLM=Mistral 7B2024.05 | 62.23 | 22.02 | 1.96 | |
| GreedySearchLLM=Mistral 7B2024.05 | 60.64 | 23.18 | 4.38 | |
| AdaptiveSearchLLM=Mistral 7B2024.05 | 54.04 | 21.35 | 9.99 | |
| SGCLLM=Vicuna 13B2024.05 | 48.79 | 15.99 | 1.89 | |
| BeamSearchLLM=Vicuna 13B2024.05 | 46.91 | 15.41 | 7.79 | |
| Greedy SearchLLM=Vicuna 13B2024.05 | 45.22 | 13.69 | 3.42 | |
| DirectLLM=Vicuna 13B2024.05 | 44.66 | 14.01 | 2.02 | |
| AdaptiveSearchLLM=Vicuna 13B2024.05 | 41.32 | 7.02 | 6.51 |