Knowledge Retrieval on STaRK-MAG synthetic (test)
78.6Hit@1AF-Retriever
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| AF-RetrieverShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 78.6 | 91.4 | 84 | |
| GraphRAFTTraining protocol=Supervised / Fine-tuned2025.05 | 71.1 | 85.3 | 76.9 | |
| HybGRAGShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 65.4 | 75.3 | 69.8 | |
| MoRTraining protocol=Supervised / Fine-tuned2025.05 | 58.2 | 78.3 | 67.1 | |
| 4StepFocusShot-count=Zero/One-shot, Test Set Scale=10%-version2025.05 | 53.8 | 69.2 | 61.4 | |
| KARShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 50.5 | 65.4 | 57.5 | |
| mFARTraining protocol=Supervised / Fine-tuned2025.05 | 49 | 69.6 | 58.2 | |
| AvaTaRShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 44.4 | 59.7 | 51.2 | |
| PASemiQATraining protocol=Supervised / Fine-tuned2025.05 | 43.2 | — | 50.2 | |
| VSS+[pointw.]RerankerShot-count=Zero/One-shot, Test Set Scale=10%-version2025.05 | 40.9 | 58.2 | 49 | |
| ReflexionShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 40.7 | 54.4 | 47.1 | |
| ReActShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 31.1 | 49.5 | 39.2 | |
| VSS (Ada-002)Shot-count=Zero/One-shot, Test Set Scale=Full, Embedding=Ada-0022025.05 | 29.1 | 49.6 | 38.6 | |
| VSS (Multi-Ada)Shot-count=Zero/One-shot, Test Set Scale=Full, Embedding=Multi-Ada2025.05 | 25.9 | 50.4 | 36.9 | |
| BM25Shot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 25.8 | 45.2 | 34.9 | |
| ToGShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 13.2 | 16.2 | 14.2 | |
| QA-GNNShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 12.9 | 39 | 29.1 | |
| DPRShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 10.5 | 35.2 | 21.3 |