Knowledge Retrieval on STaRK-AMAZON synthetic (test)
61.2Hit@1AF-Retriever
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AF-RetrieverShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 61.2 | 75.2 | 67.3 | |
| KARShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 54.2 | 68.7 | 61.3 | |
| MoRTraining protocol=Supervised / Fine-tuned2025.05 | 52.2 | 74.6 | 62.2 | |
| AvaTaRShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 49.9 | 69.2 | 58.7 | |
| 4StepFocusShot-count=Zero/One-shot, Test Set Scale=10%-version2025.05 | 47.6 | 67.6 | 56.5 | |
| PASemiQATraining protocol=Supervised / Fine-tuned2025.05 | 45.9 | — | 55.7 | |
| BM25Shot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 44.9 | 67.4 | 55.3 | |
| VSS+[pointw.]RerankerShot-count=Zero/One-shot, Test Set Scale=10%-version2025.05 | 44.8 | 71.2 | 55.7 | |
| ReflexionShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 42.8 | 65 | 52.9 | |
| ReActShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 42.1 | 64.6 | 52.3 | |
| mFARTraining protocol=Supervised / Fine-tuned2025.05 | 41.2 | 70 | 54.2 | |
| VSS (Multi-Ada)Shot-count=Zero/One-shot, Test Set Scale=Full, Embedding=Multi-Ada2025.05 | 40.1 | 65 | 51.6 | |
| VSS (Ada-002)Shot-count=Zero/One-shot, Test Set Scale=Full, Embedding=Ada-0022025.05 | 39.2 | 62.7 | 50.4 | |
| QA-GNNShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 26.6 | 50 | 37.8 | |
| DPRShot-count=Zero/One-shot, Test Set Scale=Full2025.05 | 15.3 | 47.9 | 30.2 |