Knowledge Graph Retrieval on STARK PRIME
68.9H@1GRASP
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GRASPCategory=Hybrid, Setting=Sup2026.05 | 68.9 | 88.2 | 77.7 | 89.5 | |
| w/o RR f.t.Category=Hybrid, Setting=ZS2026.05 | 61.1 | 83.6 | 70.8 | 85.4 | |
| w/o RRCategory=Hybrid, Setting=ZS2026.05 | 61.1 | 83.6 | 70.8 | 85.4 | |
| AF-RetrieverCategory=Hybrid, Setting=ZS2026.05 | 57.1 | 69.4 | 62.7 | — | |
| w/o RR & FSCategory=Hybrid, Setting=ZS2026.05 | 54.4 | 73.5 | 62.7 | 77.8 | |
| ToG + GPT-4oreranking=true2026.05 | 53.3 | 63.7 | 57.8 | 54.4 | |
| GraphFlowreranking=true2026.05 | 51.4 | 72.1 | 61.4 | 79.7 | |
| KARCategory=Hybrid, Setting=ZS2026.05 | 45 | 60.6 | 51.9 | 59.9 | |
| GraphFlowreranking=false2026.05 | 39.8 | 71.7 | 54.6 | 79.7 | |
| mFAR*Category=Hybrid, Setting=Sup2026.05 | 37.1 | 62.9 | 48.5 | 67.8 | |
| AvaTaRCategory=Hybrid, Setting=Sup2026.05 | 33 | 51.4 | 41 | — | |
| SeedERtext encoder=Qwen3-4B-Embedding2026.05 | 31 | 58.2 | 42.9 | 64.7 | |
| SFTreranking=false2026.05 | 27.5 | 40.1 | 33.1 | 47.7 | |
| multi-ada-002Category=Textual, Setting=ZS2026.05 | 24.5 | 39.8 | 33 | 47.2 | |
| SFTreranking=true2026.05 | 23.8 | 52.5 | 36 | 47.7 | |
| PRMreranking=true2026.05 | 22.9 | 28.2 | 26.9 | 46 | |
| BM25Category=Textual, Setting=ZS2026.05 | 22.5 | 41.8 | 30.4 | 42.3 | |
| ToG + LLaMA3reranking=false2026.05 | 21.9 | 34 | 26.6 | 33.8 | |
| ToG + LLaMA3reranking=true2026.05 | 21.9 | 34 | 26.6 | 33.8 | |
| PRMreranking=false2026.05 | 21 | 46.7 | 31.3 | 46 | |
| ada-002Category=Textual, Setting=ZS2026.05 | 17.4 | 34.7 | 26.4 | 41.1 | |
| ToG + GPT-4oreranking=false2026.05 | 16.7 | 39.8 | 27 | 54.4 | |
| Qwen3/SapBERTCategory=Textual, Setting=ZS2026.05 | 11.4 | 20 | 16.6 | 27.6 | |
| QAGNNCategory=Struct., Setting=ZS2026.05 | 6.1 | 13.3 | 9.4 | 17.6 | |
| DPRCategory=Textual, Setting=ZS2026.05 | 2 | 9.2 | 7.1 | 10.7 |