Knowledge Graph Retrieval on STaRK-MAG human-generated
79.7H@1GRASP
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GRASPCategory=Hybrid, Setting=Sup2026.05 | 79.7 | 91.7 | 89.6 | 85.4 | |
| w/o RR f.t.Category=Hybrid, Setting=ZS2026.05 | 67.3 | 85.3 | 87.1 | 76.2 | |
| w/o RR & FSCategory=Hybrid, Setting=ZS2026.05 | 65.6 | 81.1 | 84.2 | 72.9 | |
| w/o RRCategory=Hybrid, Setting=ZS2026.05 | 63.5 | 85 | 86.6 | 73.6 | |
| AF-RetrieverCategory=Hybrid, Setting=ZS2026.05 | 52.4 | 60.7 | — | 55.9 | |
| KARCategory=Hybrid, Setting=ZS2026.05 | 51.2 | 58.3 | 46.6 | 54.5 | |
| mFAR*Category=Hybrid, Setting=Sup2026.05 | 43.6 | 68.1 | 69 | 54.9 | |
| Qwen3/SapBERTCategory=Textual, Setting=ZS2026.05 | 35.3 | 56.4 | 56.6 | 45.5 | |
| AvaTaRCategory=Hybrid, Setting=Sup2026.05 | 33.3 | 42.9 | — | 38.6 | |
| BM25Category=Textual, Setting=ZS2026.05 | 32.1 | 41.7 | 32.5 | 37.4 | |
| ada-002Category=Textual, Setting=ZS2026.05 | 28.6 | 41.7 | 36 | 35.8 | |
| multi-ada-002Category=Textual, Setting=ZS2026.05 | 23.8 | 41.7 | 39.9 | 31.4 | |
| QAGNNCategory=Struct., Setting=ZS2026.05 | 20.2 | 26.2 | 28.8 | 25.5 | |
| DPRCategory=Textual, Setting=ZS2026.05 | 4.7 | 9.5 | 25 | 7.9 |