Task Planning on TaskBench Multimedia v1 (test)
88.54n-F1SGC
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SGCLLM=GPT 4-turbo2024.05 | 88.54 | 69.83 | 2,100 | |
| DirectLLM=GPT 4-turbo2024.05 | 88.29 | 69.38 | 2,280 | |
| SGCLLM=GPT 3.5-turbo2024.05 | 83.65 | 63.65 | 2,090 | |
| DirectLLM=GPT 3.5-turbo2024.05 | 82.85 | 62.07 | 2,260 | |
| SGCLLM=Mistral 7B2024.05 | 74.07 | 49.9 | 2,430 | |
| SGCLLM=CodeLlama 13B2024.05 | 73.32 | 53.28 | 2,430 | |
| DirectLLM=Mistral 7B2024.05 | 69.83 | 39.85 | 2,640 | |
| DirectLLM=CodeLlama 13B2024.05 | 68.57 | 41.79 | 2,590 | |
| SGCLLM=CodeLlama 7B2024.05 | 67.04 | 45.04 | 2,430 | |
| SGCLLM=Vicuna 13B2024.05 | 61.78 | 37.6 | 2,430 | |
| DirectLLM=CodeLlama 7B2024.05 | 59.44 | 30.83 | 2,570 | |
| SGCLLM=Baichuan2 13B2024.05 | 56.75 | 31.62 | 2,430 | |
| DirectLLM=Vicuna 13B2024.05 | 53.57 | 23.19 | 2,640 | |
| DirectLLM=Baichuan2 13B2024.05 | 47.57 | 4.08 | 2,590 |