Knowledge Graph Retrieval on Amazon (test)
71.2Hit@1GRASP
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GRASPCategory=Hybrid, Setting=Supervised2026.05 | 71.2 | 88.3 | 70.6 | 78.7 | |
| AF-RetrieverCategory=Hybrid, Setting=Zero-shot2026.05 | 61.2 | 75.2 | 35.5 | 67.3 | |
| GRASP w/o RR f.t.Category=Hybrid, Setting=Zero-shot, reranker=none, fine-tuning=true2026.05 | 57.4 | 83.9 | 68.3 | 68.9 | |
| KARCategory=Hybrid2025.02 | 54.2 | 68.7 | 57.24 | 61.29 | |
| KARCategory=Hybrid, Setting=Zero-shot2026.05 | 54.2 | 68.7 | 57.2 | 61.3 | |
| MoRCategory=Hybrid, Setting=Supervised2026.05 | 52.2 | 74.7 | 59.9 | 62.2 | |
| MoRCategory=Hybrid2025.02 | 52.19 | 74.65 | 59.92 | 62.24 | |
| GRASP w/o RRCategory=Hybrid, Setting=Zero-shot, reranker=none2026.05 | 51.2 | 74.5 | 63.1 | 61.9 | |
| AvaTaRCategory=Hybrid, Setting=Supervised2026.05 | 49.9 | 69.2 | 60.6 | 58.7 | |
| AvaTaRCategory=Hybrid2025.02 | 49.87 | 69.16 | 60.57 | 58.7 | |
| mFAR*Category=Hybrid, Setting=Supervised2026.05 | 48.3 | 72.1 | 58.4 | 58.9 | |
| BM25Category=Textual2025.02 | 44.94 | 67.42 | 53.77 | 55.3 | |
| BM25Category=Textual, Setting=Zero-shot2026.05 | 44.9 | 67.4 | 53.8 | 55.3 | |
| Qwen3/SapBERTCategory=Textual, Setting=Zero-shot2026.05 | 44.6 | 67.2 | 54.8 | 54.9 | |
| MoRw/o RCategory=Ablation2025.02 | 44.21 | 68.87 | 56.5 | 55.28 | |
| MoRw/o RSCategory=Ablation2025.02 | 43.05 | 69.36 | 57.38 | 54.69 | |
| MFAR*Category=Hybrid2025.02 | 41.2 | 70 | 58.5 | 54.2 | |
| mFARCategory=Hybrid, Setting=Supervised2026.05 | 41.2 | 70 | 58.5 | 54.2 | |
| multi-ada-002Category=Textual, Setting=Zero-shot2026.05 | 40.1 | 65 | 55.1 | 51.6 | |
| Multi-ada-002Category=Textual2025.02 | 40.07 | 64.98 | 55.12 | 51.55 | |
| ada-002Category=Textual, Setting=Zero-shot2026.05 | 39.2 | 62.7 | 53.3 | 50.4 | |
| Ada-002Category=Textual2025.02 | 39.16 | 62.73 | 53.29 | 50.35 | |
| GRASP w/o RR & FSCategory=Hybrid, Setting=Zero-shot, reranker=none, fusion=none2026.05 | 37.2 | 55.4 | 45.3 | 45.6 | |
| MoRw/o RTCategory=Ablation2025.02 | 34.04 | 53.41 | 45.16 | 42.85 | |
| QAGNNCategory=Structural, Setting=Zero-shot2026.05 | 26.6 | 50 | 52.1 | 37.8 | |
| QAGNNCategory=Structural (KG)2025.02 | 26.56 | 50.01 | 52.05 | 37.75 | |
| DPRCategory=Textual, Setting=Zero-shot2026.05 | 15.3 | 47.9 | 44.5 | 30.2 | |
| DPRCategory=Textual2025.02 | 15.29 | 47.93 | 44.49 | 30.2 |