Graph Understanding on GraphSQA Micro-level Tasks 1.0 (test)
94.01ScoreGraphInsight
Evaluation Results
| Method | Links | |
|---|---|---|
| GraphInsightModel=Llama-3-8B-262k, Method Category=Proposed2024.09 | 94.01 | |
| GraphInsightModel=Qwen2-7B, Method Category=Proposed2024.09 | 83.87 | |
| Raw Seq.Model=Llama-3-8B-262k, Method Category=Baseline2024.09 | 77.86 | |
| DFSModel=Llama-3-8B-262k, Method Category=Reordering Methods2024.09 | 77.57 | |
| BFSModel=Llama-3-8B-262k, Method Category=Reordering Methods2024.09 | 77.52 | |
| SPModel=Llama-3-8B-262k, Method Category=Reordering Methods2024.09 | 77.5 | |
| GraphInsightModel=Llama-3-8B, Method Category=Proposed2024.09 | 76.05 | |
| GraphInsightModel=Mistral-7B, Method Category=Proposed2024.09 | 74.25 | |
| ALModel=Llama-3-8B-262k, Method Category=Structural Methods2024.09 | 74.22 | |
| BAGModel=Llama-3-8B-262k, Method Category=Prompting Methods2024.09 | 69.28 | |
| FSModel=Llama-3-8B-262k, Method Category=Prompting Methods2024.09 | 65.59 | |
| AMModel=Llama-3-8B-262k, Method Category=Structural Methods2024.09 | 59.82 | |
| COTModel=Mistral-7B, Method Category=Prompting Methods2024.09 | 59.35 | |
| ALModel=Qwen2-7B, Method Category=Structural Methods2024.09 | 58.28 | |
| COTModel=Qwen2-7B, Method Category=Prompting Methods2024.09 | 58.13 | |
| BAGModel=Mistral-7B, Method Category=Prompting Methods2024.09 | 57.05 | |
| DFSModel=Mistral-7B, Method Category=Reordering Methods2024.09 | 56.55 | |
| SPModel=Mistral-7B, Method Category=Reordering Methods2024.09 | 56.51 | |
| BFSModel=Mistral-7B, Method Category=Reordering Methods2024.09 | 56.5 | |
| Raw Seq.Model=Qwen2-7B, Method Category=Baseline2024.09 | 56.37 | |
| Raw Seq.Model=Mistral-7B, Method Category=Baseline2024.09 | 56.35 | |
| ALModel=Mistral-7B, Method Category=Structural Methods2024.09 | 55.19 | |
| FSModel=Qwen2-7B, Method Category=Prompting Methods2024.09 | 54.21 | |
| BAGModel=Qwen2-7B, Method Category=Prompting Methods2024.09 | 51.96 | |
| COTModel=Llama-3-8B-262k, Method Category=Prompting Methods2024.09 | 49.85 | |
| BFSModel=Qwen2-7B, Method Category=Reordering Methods2024.09 | 49.84 | |
| DFSModel=Qwen2-7B, Method Category=Reordering Methods2024.09 | 49.01 | |
| SPModel=Qwen2-7B, Method Category=Reordering Methods2024.09 | 48.76 | |
| DFSModel=Llama-3-8B, Method Category=Reordering Methods2024.09 | 47.26 | |
| COTModel=Llama-3-8B, Method Category=Prompting Methods2024.09 | 47.02 | |
| BFSModel=Llama-3-8B, Method Category=Reordering Methods2024.09 | 46.58 | |
| SPModel=Llama-3-8B, Method Category=Reordering Methods2024.09 | 46.56 | |
| Raw Seq.Model=Llama-3-8B, Method Category=Baseline2024.09 | 45.97 | |
| BAGModel=Llama-3-8B, Method Category=Prompting Methods2024.09 | 45.45 | |
| FSModel=Llama-3-8B, Method Category=Prompting Methods2024.09 | 44.68 | |
| GraphInsightModel=Vicuna-7B, Method Category=Proposed2024.09 | 43.46 | |
| FSModel=Mistral-7B, Method Category=Prompting Methods2024.09 | 41.92 | |
| ALModel=Llama-3-8B, Method Category=Structural Methods2024.09 | 38.69 | |
| GraphTokenModel=Llama-3-8B, Method Category=Baseline2024.09 | 31.61 | |
| GraphTokenModel=Qwen2-7B, Method Category=Baseline2024.09 | 29.6 | |
| AMModel=Qwen2-7B, Method Category=Structural Methods2024.09 | 22.62 | |
| GraphTokenModel=Mistral-7B, Method Category=Baseline2024.09 | 22.08 | |
| GraphTokenModel=Vicuna-7B, Method Category=Baseline2024.09 | 20.16 | |
| AMModel=Mistral-7B, Method Category=Structural Methods2024.09 | 19.94 | |
| GraphTokenModel=Llama-3-8B-262k, Method Category=Baseline2024.09 | 17.7 | |
| AMModel=Llama-3-8B, Method Category=Structural Methods2024.09 | 15.81 | |
| DFSModel=Vicuna-7B, Method Category=Reordering Methods2024.09 | 12.89 | |
| SPModel=Vicuna-7B, Method Category=Reordering Methods2024.09 | 12.85 | |
| BFSModel=Vicuna-7B, Method Category=Reordering Methods2024.09 | 12.82 | |
| Raw Seq.Model=Vicuna-7B, Method Category=Baseline2024.09 | 12.76 | |
| FSModel=Vicuna-7B, Method Category=Prompting Methods2024.09 | 11.44 | |
| COTModel=Vicuna-7B, Method Category=Prompting Methods2024.09 | 11.37 | |
| BAGModel=Vicuna-7B, Method Category=Prompting Methods2024.09 | 11.11 | |
| ALModel=Vicuna-7B, Method Category=Structural Methods2024.09 | 10.46 | |
| AMModel=Vicuna-7B, Method Category=Structural Methods2024.09 | 3.06 |