Multi-step Retrieval on 2WikiMultihopQA (val)
68.02F1 ScoreGritLM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GritLMModel Category=Dense Models, top_k=32026.02 | 68.02 | — | — | |
| GTE-Qwen2Model Category=Dense Models, top_k=32026.02 | 66.4 | — | — | |
| ANCEModel Category=Dense Models, top_k=32026.02 | 66.03 | — | — | |
| Qwen3Model Category=Dense Models, top_k=32026.02 | 65.33 | — | — | |
| AttentionRetrieverModel Category=AttentionRetriever (Ours), Backbone=LLaMA-3.2, Model size=3B, top_k=32026.02 | 64.95 | — | — | |
| GTRModel Category=Dense Models, top_k=32026.02 | 63.93 | — | — | |
| AttentionRetrieverModel Category=AttentionRetriever (Ours), Backbone=Qwen-2.5, Model size=3B, top_k=32026.02 | 63.55 | — | — | |
| SPScannerModel Category=Autoregressive Models, top_k=32026.02 | 62.68 | — | — | |
| BM25Model Category=Sparse Models, top_k=32026.02 | 54.54 | — | — | |
| DPRModel Category=Dense Models, top_k=32026.02 | 53.89 | — | — | |
| CDEModel Category=Dense Models, top_k=32026.02 | 26.37 | — | — | |
| IRCoT + BM25 (Default)Framework=IRCoT, Retriever=BM252025.12 | — | 61.2 | 75.6 | |
| IRCoT + ColBERTv2Framework=IRCoT, Retriever=ColBERTv22025.12 | — | 64.1 | 74.4 | |
| IRCoT + ContrieverFramework=IRCoT, Retriever=Contriever2025.12 | — | 51.6 | 63.8 | |
| IRCoT + CoopRAG (GPT-4o-mini)Framework=IRCoT, Retriever=CoopRAG, Model=GPT-4o-mini2025.12 | — | 81 | 95.9 | |
| IRCoT + HippoRAG (ColBERTv2)Framework=IRCoT, Retriever=HippoRAG, Backbone=ColBERTv22025.12 | — | 75.8 | 93.9 | |
| IRCoT + HippoRAG (Contriever)Framework=IRCoT, Retriever=HippoRAG, Backbone=Contriever2025.12 | — | 75.3 | 93.4 | |
| KeyExtract + CoopRAG (Gemma2-9B)Framework=KeyExtract, Retriever=CoopRAG, Model=Gemma2-9B2025.12 | — | 81.8 | 97.2 | |
| KeyExtract + CoopRAG (GPT-4o-mini)Framework=KeyExtract, Retriever=CoopRAG, Model=GPT-4o-mini2025.12 | — | 83.6 | 97.6 |