Information Retrieval on HotpotQA 2018 (test)
86.9Recall@4ACQO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ACQORetriever=Sparse (BM25), Backbone=Qwen2.5-3B2026.01 | 86.9 | 91.6 | 97.5 | 51.2 | 77.7 | 74.2 | |
| RawRetriever=Sparse (BM25), query_decomposition=none2026.01 | 83.3 | 88.9 | 96.7 | 49.5 | 75.4 | 70.5 | |
| ConvSearch-R1Retriever=Sparse (BM25), Backbone=Qwen2.5-3B2026.01 | 83 | 90.2 | 96 | 51.1 | 77 | 72.3 | |
| vanilla RLRetriever=Sparse (BM25), Backbone=Qwen2.5-3B2026.01 | 82.3 | 89.9 | 95.6 | 48.8 | 77.5 | 73.2 | |
| ACQORetriever=Dense (ANCE), Backbone=Qwen2.5-3B2026.01 | 82.2 | 85.8 | 91.2 | 49.6 | 73.4 | 73.6 | |
| DeepSeek-V3.1Retriever=Sparse (BM25), query_decomposition=with2026.01 | 81.1 | 86.6 | 93.3 | 49.1 | 70.6 | 66.2 | |
| Qwen2.5-3B-instRetriever=Sparse (BM25), query_decomposition=with2026.01 | 75.3 | 81.2 | 89.5 | 42.7 | 65.9 | 62.4 | |
| Qwen2.5-3B-instRetriever=Sparse (BM25), query_decomposition=without2026.01 | 72 | 79.3 | 89.7 | 41.2 | 64.2 | 60.5 | |
| RawRetriever=Dense (ANCE), query_decomposition=none2026.01 | 68.3 | 74.8 | 86.1 | 34.8 | 60.4 | 59.5 |