Question Answering Retrieval on Natural Questions NQ320k
0.685HR@1MoL
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| MoLRetrieval Strategy=Learned similarities, Backbone=T5-base2024.07 | 0.685 | 0.919 | 0.97 | 0.773 | |
| GenRetRetrieval Strategy=Generative retrieval, Backbone=T5-base2024.07 | 0.681 | 0.888 | 0.952 | 0.759 | |
| MoL abl. LMIRetrieval Strategy=Learned similarities, Backbone=T5-base, Ablation=Without load balancing loss (LMI)2024.07 | 0.673 | 0.919 | 0.968 | 0.767 | |
| NCIRetrieval Strategy=Generative retrieval, Backbone=T5-base2024.07 | 0.659 | 0.852 | 0.924 | 0.731 | |
| DSI+QGRetrieval Strategy=Generative retrieval, Backbone=T5-base2024.07 | 0.631 | 0.807 | 0.88 | 0.695 | |
| SEALRetrieval Strategy=Generative retrieval, Backbone=T5-base2024.07 | 0.57 | 0.8 | 0.914 | 0.655 | |
| GTR-BaseRetrieval Strategy=Dense retrieval2024.07 | 0.56 | 0.844 | 0.937 | 0.662 | |
| GENRERetrieval Strategy=Generative retrieval, Backbone=T5-base2024.07 | 0.552 | 0.673 | 0.754 | 0.599 | |
| DSIRetrieval Strategy=Generative retrieval, Backbone=T5-base2024.07 | 0.552 | 0.674 | 0.78 | 0.596 | |
| Sentence-T5Retrieval Strategy=Dense retrieval, Backbone=T5-base2024.07 | 0.536 | 0.83 | 0.938 | 0.641 | |
| DPRRetrieval Strategy=Dense retrieval2024.07 | 0.502 | 0.777 | 0.909 | 0.599 | |
| DocT5QueryRetrieval Strategy=Sparse retrieval2024.07 | 0.38 | 0.693 | 0.861 | 0.489 | |
| BM25Retrieval Strategy=Sparse retrieval2024.07 | 0.297 | 0.603 | 0.821 | 0.402 |