Graph Understanding on GraphSQA Macro-level Tasks 1.0 (test)
69.28ScoreGraphInsight
Evaluation Results
| Method | Links | |
|---|---|---|
| GraphInsightModel=Llama-3-8B-262k, Method Category=Proposed2024.09 | 69.28 | |
| GraphInsightModel=Qwen2-7B, Method Category=Proposed2024.09 | 65.87 | |
| BAGModel=Qwen2-7B, Method Category=Prompting Methods2024.09 | 64.47 | |
| DFSModel=Llama-3-8B-262k, Method Category=Reordering Methods2024.09 | 62.21 | |
| Raw Seq.Model=Llama-3-8B-262k, Method Category=Baseline2024.09 | 62.18 | |
| BFSModel=Llama-3-8B-262k, Method Category=Reordering Methods2024.09 | 61.89 | |
| SPModel=Llama-3-8B-262k, Method Category=Reordering Methods2024.09 | 61.83 | |
| BFSModel=Qwen2-7B, Method Category=Reordering Methods2024.09 | 60.17 | |
| SPModel=Qwen2-7B, Method Category=Reordering Methods2024.09 | 59.58 | |
| DFSModel=Qwen2-7B, Method Category=Reordering Methods2024.09 | 59.13 | |
| GraphInsightModel=Mistral-7B, Method Category=Proposed2024.09 | 58.46 | |
| COTModel=Llama-3-8B-262k, Method Category=Prompting Methods2024.09 | 57.51 | |
| Raw Seq.Model=Qwen2-7B, Method Category=Baseline2024.09 | 56.44 | |
| FSModel=Mistral-7B, Method Category=Prompting Methods2024.09 | 54.39 | |
| COTModel=Qwen2-7B, Method Category=Prompting Methods2024.09 | 54.23 | |
| GraphInsightModel=Llama-3-8B, Method Category=Proposed2024.09 | 54.22 | |
| FSModel=Qwen2-7B, Method Category=Prompting Methods2024.09 | 53.66 | |
| ALModel=Llama-3-8B-262k, Method Category=Structural Methods2024.09 | 52.92 | |
| ALModel=Qwen2-7B, Method Category=Structural Methods2024.09 | 52.23 | |
| Raw Seq.Model=Mistral-7B, Method Category=Baseline2024.09 | 52.22 | |
| SPModel=Mistral-7B, Method Category=Reordering Methods2024.09 | 52.22 | |
| BFSModel=Mistral-7B, Method Category=Reordering Methods2024.09 | 52.12 | |
| DFSModel=Mistral-7B, Method Category=Reordering Methods2024.09 | 52.05 | |
| BAGModel=Llama-3-8B-262k, Method Category=Prompting Methods2024.09 | 51.15 | |
| ALModel=Mistral-7B, Method Category=Structural Methods2024.09 | 50.01 | |
| FSModel=Llama-3-8B-262k, Method Category=Prompting Methods2024.09 | 46.52 | |
| GraphTokenModel=Qwen2-7B, Method Category=Baseline2024.09 | 44.93 | |
| SPModel=Llama-3-8B, Method Category=Reordering Methods2024.09 | 44.04 | |
| BFSModel=Llama-3-8B, Method Category=Reordering Methods2024.09 | 43.81 | |
| Raw Seq.Model=Llama-3-8B, Method Category=Baseline2024.09 | 43.79 | |
| DFSModel=Llama-3-8B, Method Category=Reordering Methods2024.09 | 43.76 | |
| ALModel=Llama-3-8B, Method Category=Structural Methods2024.09 | 43.36 | |
| COTModel=Mistral-7B, Method Category=Prompting Methods2024.09 | 43.23 | |
| BAGModel=Mistral-7B, Method Category=Prompting Methods2024.09 | 42.49 | |
| GraphTokenModel=Llama-3-8B-262k, Method Category=Baseline2024.09 | 41.98 | |
| COTModel=Llama-3-8B, Method Category=Prompting Methods2024.09 | 41.87 | |
| BAGModel=Llama-3-8B, Method Category=Prompting Methods2024.09 | 40.6 | |
| GraphTokenModel=Llama-3-8B, Method Category=Baseline2024.09 | 39.89 | |
| FSModel=Llama-3-8B, Method Category=Prompting Methods2024.09 | 37.02 | |
| AMModel=Llama-3-8B-262k, Method Category=Structural Methods2024.09 | 35.29 | |
| GraphTokenModel=Mistral-7B, Method Category=Baseline2024.09 | 33.42 | |
| GraphInsightModel=Vicuna-7B, Method Category=Proposed2024.09 | 22.96 | |
| AMModel=Qwen2-7B, Method Category=Structural Methods2024.09 | 19.15 | |
| AMModel=Mistral-7B, Method Category=Structural Methods2024.09 | 18.08 | |
| GraphTokenModel=Vicuna-7B, Method Category=Baseline2024.09 | 15.75 | |
| Raw Seq.Model=Vicuna-7B, Method Category=Baseline2024.09 | 12.67 | |
| BFSModel=Vicuna-7B, Method Category=Reordering Methods2024.09 | 12.66 | |
| SPModel=Vicuna-7B, Method Category=Reordering Methods2024.09 | 12.63 | |
| DFSModel=Vicuna-7B, Method Category=Reordering Methods2024.09 | 12.59 | |
| AMModel=Llama-3-8B, Method Category=Structural Methods2024.09 | 11.98 | |
| COTModel=Vicuna-7B, Method Category=Prompting Methods2024.09 | 11.64 | |
| BAGModel=Vicuna-7B, Method Category=Prompting Methods2024.09 | 9.95 | |
| FSModel=Vicuna-7B, Method Category=Prompting Methods2024.09 | 8.77 | |
| ALModel=Vicuna-7B, Method Category=Structural Methods2024.09 | 5.67 | |
| AMModel=Vicuna-7B, Method Category=Structural Methods2024.09 | 4.09 |