Task Planning on TaskBench Multimedia
88.54Node F1SGC
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SGCLLM=GPT-4-turbo2024.05 | 88.54 | 69.83 | 2.1 | |
| DirectLLM=GPT-4-turbo2024.05 | 88.29 | 69.38 | 2.28 | |
| BeamSearchLLM=GPT-4-turbo2024.05 | 88.16 | 70.39 | 6.9 | |
| AdaptiveSearchLLM=GPT-4-turbo2024.05 | 88.02 | 69.99 | 7.14 | |
| Greedy SearchLLM=GPT-4-turbo2024.05 | 86.81 | 69.8 | 5.52 | |
| SGCLLM=GPT-3.5-turbo2024.05 | 83.65 | 63.65 | 2.09 | |
| BeamSearchLLM=GPT-3.5-turbo2024.05 | 83.57 | 64.5 | 12.91 | |
| DirectLLM=GPT-3.5-turbo2024.05 | 82.85 | 62.07 | 2.26 | |
| AdaptiveSearchLLM=GPT-3.5-turbo2024.05 | 81.86 | 62.71 | 5.71 | |
| Greedy SearchLLM=GPT-3.5-turbo2024.05 | 81.11 | 63.02 | 4.92 | |
| SGCLLM=Mistral 7B2024.05 | 74.07 | 49.9 | 2.43 | |
| BeamSearchLLM=Mistral 7B2024.05 | 73.55 | 47.12 | 31.1 | |
| SGCLLM=CodeLlama 13B2024.05 | 73.32 | 53.28 | 2.43 | |
| AdaptiveSearchLLM=Mistral 7B2024.05 | 71.59 | 44.84 | 10.66 | |
| DirectLLM=Mistral 7B2024.05 | 69.83 | 39.85 | 2.64 | |
| BeamSearchLLM=CodeLlama 13B2024.05 | 69.53 | 43.35 | 19.51 | |
| DirectLLM=CodeLlama 13B2024.05 | 68.57 | 41.79 | 2.59 | |
| AdaptiveSearchLLM=CodeLlama 13B2024.05 | 68.14 | 41.71 | 6.77 | |
| Greedy SearchLLM=CodeLlama 13B2024.05 | 67.98 | 42.04 | 4.95 | |
| SGCLLM=Vicuna 13B2024.05 | 61.78 | 37.6 | 2.43 | |
| GreedySearchLLM=Mistral 7B2024.05 | 58.92 | 34.72 | 6.26 | |
| BeamSearchLLM=Vicuna 13B2024.05 | 54.09 | 26.19 | 25.42 | |
| DirectLLM=Vicuna 13B2024.05 | 53.57 | 23.19 | 2.64 | |
| AdaptiveSearchLLM=Vicuna 13B2024.05 | 51.24 | 24.32 | 11.25 | |
| Greedy SearchLLM=Vicuna 13B2024.05 | 46.99 | 23.11 | 5.55 |