Graph Understanding on GraphSQA Overall 1.0 (test)
82.85ScoreGraphInsight
Evaluation Results
| Method | Links | |
|---|---|---|
| GraphInsightModel=Llama-3-8B-262k, Method Category=Proposed2024.09 | 82.85 | |
| GraphInsightModel=Qwen2-7B, Method Category=Proposed2024.09 | 76.95 | |
| Raw Seq.Model=Llama-3-8B-262k, Method Category=Baseline2024.09 | 71.83 | |
| DFSModel=Llama-3-8B-262k, Method Category=Reordering Methods2024.09 | 71.66 | |
| BFSModel=Llama-3-8B-262k, Method Category=Reordering Methods2024.09 | 71.5 | |
| SPModel=Llama-3-8B-262k, Method Category=Reordering Methods2024.09 | 71.47 | |
| GraphInsightModel=Mistral-7B, Method Category=Proposed2024.09 | 68.11 | |
| GraphInsightModel=Llama-3-8B, Method Category=Proposed2024.09 | 67.65 | |
| ALModel=Llama-3-8B-262k, Method Category=Structural Methods2024.09 | 66.03 | |
| BAGModel=Llama-3-8B-262k, Method Category=Prompting Methods2024.09 | 62.31 | |
| FSModel=Llama-3-8B-262k, Method Category=Prompting Methods2024.09 | 58.25 | |
| BAGModel=Qwen2-7B, Method Category=Prompting Methods2024.09 | 56.77 | |
| COTModel=Qwen2-7B, Method Category=Prompting Methods2024.09 | 56.63 | |
| Raw Seq.Model=Qwen2-7B, Method Category=Baseline2024.09 | 56.4 | |
| ALModel=Qwen2-7B, Method Category=Structural Methods2024.09 | 55.95 | |
| SPModel=Mistral-7B, Method Category=Reordering Methods2024.09 | 54.86 | |
| DFSModel=Mistral-7B, Method Category=Reordering Methods2024.09 | 54.82 | |
| BFSModel=Mistral-7B, Method Category=Reordering Methods2024.09 | 54.81 | |
| Raw Seq.Model=Mistral-7B, Method Category=Baseline2024.09 | 54.76 | |
| FSModel=Qwen2-7B, Method Category=Prompting Methods2024.09 | 54 | |
| BFSModel=Qwen2-7B, Method Category=Reordering Methods2024.09 | 53.81 | |
| ALModel=Mistral-7B, Method Category=Structural Methods2024.09 | 53.2 | |
| COTModel=Mistral-7B, Method Category=Prompting Methods2024.09 | 53.15 | |
| SPModel=Qwen2-7B, Method Category=Reordering Methods2024.09 | 52.92 | |
| DFSModel=Qwen2-7B, Method Category=Reordering Methods2024.09 | 52.9 | |
| COTModel=Llama-3-8B-262k, Method Category=Prompting Methods2024.09 | 52.79 | |
| BAGModel=Mistral-7B, Method Category=Prompting Methods2024.09 | 51.45 | |
| AMModel=Llama-3-8B-262k, Method Category=Structural Methods2024.09 | 50.38 | |
| FSModel=Mistral-7B, Method Category=Prompting Methods2024.09 | 46.71 | |
| DFSModel=Llama-3-8B, Method Category=Reordering Methods2024.09 | 45.91 | |
| SPModel=Llama-3-8B, Method Category=Reordering Methods2024.09 | 45.59 | |
| BFSModel=Llama-3-8B, Method Category=Reordering Methods2024.09 | 45.51 | |
| Raw Seq.Model=Llama-3-8B, Method Category=Baseline2024.09 | 45.13 | |
| COTModel=Llama-3-8B, Method Category=Prompting Methods2024.09 | 45.04 | |
| BAGModel=Llama-3-8B, Method Category=Prompting Methods2024.09 | 43.58 | |
| FSModel=Llama-3-8B, Method Category=Prompting Methods2024.09 | 41.73 | |
| ALModel=Llama-3-8B, Method Category=Structural Methods2024.09 | 40.48 | |
| GraphInsightModel=Vicuna-7B, Method Category=Proposed2024.09 | 35.57 | |
| GraphTokenModel=Qwen2-7B, Method Category=Baseline2024.09 | 35.5 | |
| GraphTokenModel=Llama-3-8B, Method Category=Baseline2024.09 | 34.8 | |
| GraphTokenModel=Llama-3-8B-262k, Method Category=Baseline2024.09 | 27.03 | |
| GraphTokenModel=Mistral-7B, Method Category=Baseline2024.09 | 26.44 | |
| AMModel=Qwen2-7B, Method Category=Structural Methods2024.09 | 21.29 | |
| AMModel=Mistral-7B, Method Category=Structural Methods2024.09 | 19.22 | |
| GraphTokenModel=Vicuna-7B, Method Category=Baseline2024.09 | 17.94 | |
| AMModel=Llama-3-8B, Method Category=Structural Methods2024.09 | 14.34 | |
| DFSModel=Vicuna-7B, Method Category=Reordering Methods2024.09 | 12.77 | |
| SPModel=Vicuna-7B, Method Category=Reordering Methods2024.09 | 12.77 | |
| BFSModel=Vicuna-7B, Method Category=Reordering Methods2024.09 | 12.76 | |
| Raw Seq.Model=Vicuna-7B, Method Category=Baseline2024.09 | 12.73 | |
| COTModel=Vicuna-7B, Method Category=Prompting Methods2024.09 | 11.47 | |
| BAGModel=Vicuna-7B, Method Category=Prompting Methods2024.09 | 10.66 | |
| FSModel=Vicuna-7B, Method Category=Prompting Methods2024.09 | 10.41 | |
| ALModel=Vicuna-7B, Method Category=Structural Methods2024.09 | 8.62 | |
| AMModel=Vicuna-7B, Method Category=Structural Methods2024.09 | 3.45 |