Task Parameter Prediction on HuggingFace dataset TaskBench
73.09t-F1GraphSAGE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GraphSAGELLM=GPT-4-turbo, GNN Planning=Training-required GraphSAGE2024.05 | 73.09 | 58.2 | |
| SGCLLM=GPT-4-turbo, GNN Planning=Training-free SGC2024.05 | 72.91 | 58.02 | |
| GraphSAGELLM=GPT-3.5-turbo, GNN Planning=Training-required GraphSAGE2024.05 | 71.19 | 56.83 | |
| DirectLLM=GPT-4-turbo, GNN Planning=None2024.05 | 70.73 | 55.54 | |
| SGCLLM=GPT-3.5-turbo, GNN Planning=Training-free SGC2024.05 | 68.13 | 54.34 | |
| DirectLLM=GPT-3.5-turbo, GNN Planning=None2024.05 | 62.42 | 48.27 | |
| GraphSAGELLM=CodeLlama-13B, GNN Planning=Training-required GraphSAGE2024.05 | 59.49 | 45.54 | |
| GraphSAGELLM=Mistral-7B, GNN Planning=Training-required GraphSAGE2024.05 | 59.07 | 41.4 | |
| SGCLLM=Mistral-7B, GNN Planning=Training-free SGC2024.05 | 58.13 | 39.64 | |
| SGCLLM=CodeLlama-13B, GNN Planning=Training-free SGC2024.05 | 57.74 | 43.21 | |
| DirectLLM=CodeLlama-13B, GNN Planning=None2024.05 | 44.62 | 33.24 | |
| DirectLLM=Mistral-7B, GNN Planning=None2024.05 | 38.77 | 18.56 |