Task Planning on Hugging Face v1 (test)
77.79n-F1SGC
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SGCLLM=GPT 4-turbo2024.05 | 77.79 | 52.2 | 2,030 | |
| DirectLLM=GPT 4-turbo2024.05 | 77.6 | 52.18 | 2,190 | |
| SGCLLM=GPT 3.5-turbo2024.05 | 76.37 | 50.04 | 2,020 | |
| DirectLLM=GPT 3.5-turbo2024.05 | 73.85 | 45.73 | 2,140 | |
| SGCLLM=Mistral 7B2024.05 | 67.43 | 42.08 | 2,320 | |
| SGCLLM=CodeLlama 13B2024.05 | 65.51 | 39.44 | 2,310 | |
| SGCLLM=CodeLlama 7B2024.05 | 63.98 | 39.27 | 2,300 | |
| DirectLLM=Mistral 7B2024.05 | 60.6 | 30.23 | 2,490 | |
| SGCLLM=Vicuna 13B2024.05 | 59.62 | 31.98 | 2,310 | |
| DirectLLM=CodeLlama 7B2024.05 | 58.06 | 29.39 | 2,440 | |
| DirectLLM=CodeLlama 13B2024.05 | 57.55 | 28.88 | 2,450 | |
| SGCLLM=Baichuan2 13B2024.05 | 56.53 | 29.94 | 2,280 | |
| DirectLLM=Vicuna 13B2024.05 | 50.46 | 21.27 | 2,500 | |
| DirectLLM=Baichuan2 13B2024.05 | 45.85 | 19 | 2,430 | |
| BeamSearchLLM=Baichuan2 13B2024.05 | 41.06 | 9.59 | 24,690 | |
| AdaptiveSearchLLM=Baichuan2 13B2024.05 | 39.3 | 10.41 | 10,810 | |
| GreedySearchLLM=Baichuan2 13B2024.05 | 30.58 | 4.89 | 6,420 |