Information Retrieval on EntityQuestions (test)
74.5Recall@20 (macro average)SPAR-PAQ (PAQ A + DPR-multi)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SPAR-PAQ (PAQ A + DPR-multi)Modeling=Dense, # LM=4, Retrieval training=Supervised + Supervised2023.03 | 74.5 | — | |
| SPAR-Wiki (Wiki A + DPR-multi)Modeling=Dense, # LM=4, Retrieval training=Supervised + Supervised2023.03 | 73.9 | — | |
| BM25Modeling=Sparse2023.03 | 72 | — | |
| PAQ AModeling=Dense, # LM=2, Retrieval training=Supervised2023.03 | 69.9 | — | |
| Wiki AModeling=Dense, # LM=2, Retrieval training=Supervised2023.03 | 69 | — | |
| Zero-NeRModeling=Dense, # LM=1, Retrieval training=None (only LM pre-training)2023.03 | 67.1 | — | |
| ContrieverModeling=Dense, # LM=2, Retrieval training=Self-supervised2023.03 | 64.7 | — | |
| DPR-PAQ (SPAR checkpoint)Modeling=Dense, # LM=2, Retrieval training=Supervised2023.03 | 63 | — | |
| DPR-PAQ (Sciavolino et al. 2021)Modeling=Dense, # LM=2, Retrieval training=Supervised2023.03 | 59.3 | — | |
| DPR-multiModeling=Dense, # LM=2, Retrieval training=Supervised2023.03 | 56.7 | — | |
| DPR-NQModeling=Dense, # LM=2, Retrieval training=Supervised2023.03 | 45.1 | — | |
| BM25retrieval unit=passages2021.10 | — | 71.2 | |
| DPRretrieval unit=passages2021.10 | — | 65.4 | |
| TOME-1retrieval unit=mentions, number of parameters=220M, number of encoded passages=12021.10 | — | 83.3 | |
| TOME-2retrieval unit=mentions, number of parameters=220M, number of encoded passages=12021.10 | — | 83.8 |