Retrieval on MS MARCO (dev)
0.401MRR@10ERNIE-Search
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ERNIE-SearchPLM (Size)=ERNIE_base, Teacher (Size)=ERNIE_large, Co-Training=true2022.06 | 0.401 | — | 0.877 | — | — | — | |
| R2ANKER (Ours)PLM (Size)=coCondenser, Teacher (Size)=R2ANKER_base, Co-Training=false2022.06 | 0.4 | — | 0.876 | — | — | — | |
| COT-MAERep type=dense, Pre-training=true, Distillation=false, Target Corpus=false, Supervised=true2023.02 | 0.399 | 0.985 | — | — | — | — | |
| ColBERTv2Retriever Type=Dense Retriever2024.11 | 0.397 | 0.984 | — | — | — | — | |
| ColBERTv2Rep type=mul-vec, Pre-training=false, Distillation=true, Target Corpus=false, Supervised=true2023.02 | 0.397 | 0.984 | — | — | — | — | |
| AR2PLM (Size)=coCondenser, Teacher (Size)=ERNIE_large, Co-Training=true2022.06 | 0.395 | — | 0.878 | — | — | — | |
| DRAGONRep type=dense, Pre-training=false, Distillation=true, Target Corpus=false, Supervised=true2023.02 | 0.393 | 0.985 | — | — | — | — | |
| DRAGON-QRep type=dense, Pre-training=false, Distillation=true, Target Corpus=false, Supervised=true2023.02 | 0.391 | 0.988 | — | — | — | — | |
| DRAGON+Rep type=dense, Pre-training=true, Distillation=true, Target Corpus=false, Supervised=true2023.02 | 0.39 | 0.986 | — | — | — | — | |
| SPLADE++Rep type=sparse, Pre-training=true, Distillation=true, Target Corpus=false, Supervised=true2023.02 | 0.389 | 0.982 | — | — | — | — | |
| RocketQAv2Retriever Type=Dense Retriever2024.11 | 0.388 | 0.981 | — | — | — | — | |
| RocketQAv2PLM (Size)=ERNIE_base, Teacher (Size)=ERNIE_base, Co-Training=true2022.06 | 0.388 | — | 0.862 | — | — | — | |
| GTR-XXLRep type=dense, Pre-training=true, Distillation=true, Target Corpus=false, Supervised=true2023.02 | 0.388 | 0.99 | — | — | — | — | |
| RocketQAv2Rep type=dense, Pre-training=false, Distillation=true, Target Corpus=false, Supervised=true2023.02 | 0.388 | 0.981 | — | — | — | — | |
| SPLADE-v3-DistilBERTRetriever Type=Sparse Retriever2024.11 | 0.387 | — | — | — | — | — | |
| coCondenserRep type=dense, Pre-training=true, Distillation=false, Target Corpus=false, Supervised=true2023.02 | 0.386 | 0.984 | — | — | — | — | |
| CoCondenserRetriever Type=Dense Retriever2024.11 | 0.382 | 0.984 | — | — | — | — | |
| coCondenserPLM (Size)=BERT_base, Co-Training=false2022.06 | 0.382 | — | — | — | — | — | |
| CL-DRDRep type=dense, Pre-training=false, Distillation=true, Target Corpus=false, Supervised=true2023.02 | 0.381 | 0.979 | — | — | — | — | |
| DRAGON-SRep type=dense, Pre-training=false, Distillation=true, Target Corpus=false, Supervised=true2023.02 | 0.381 | 0.983 | — | — | — | — | |
| PAIRPLM (Size)=ERNIE_base, Co-Training=false2022.06 | 0.379 | — | 0.864 | — | — | — | |
| SPLADE-v3-DocRetriever Type=Inference-free Sparse Retriever2024.11 | 0.378 | — | — | — | — | — | |
| Our ModelRetriever Type=Inference-free Sparse Retriever, Evaluated by Current Authors=true2024.11 | 0.378 | 0.975 | — | — | — | — | |
| RocketQARetriever Type=Dense Retriever2024.11 | 0.37 | 0.979 | — | — | — | — | |
| RocketQAPLM (Size)=ERNIE_base, Teacher (Size)=ERNIE_base, Co-Training=false2022.06 | 0.37 | — | 0.855 | — | — | — | |
| DistilSPLADE-maxRetriever Type=Sparse Retriever2024.11 | 0.368 | 0.979 | — | — | — | — | |
| CondenserPLM (Size)=BERT_base, Co-Training=false2022.06 | 0.366 | — | — | — | — | — | |
| SPLADE-doc-distillRetriever Type=Inference-free Sparse Retriever, Evaluated by Current Authors=true2024.11 | 0.365 | 0.969 | — | — | — | — | |
| BERT2019.03 | 0.365 | — | — | — | — | — | |
| ColBERTPLM (Size)=BERT_base, Co-Training=false2022.06 | 0.36 | — | 0.829 | — | — | — | |
| TCT-ColBERTRetriever Type=Dense Retriever2024.11 | 0.359 | 0.97 | — | — | — | — | |
| COCO-DRRep type=dense, Pre-training=true, Distillation=false, Target Corpus=true, Supervised=true2023.02 | 0.358 | 0.979 | — | — | — | — | |
| RetroMAEFine-tuning strategy=DPR2022.05 | 0.3553 | 0.9763 | — | 0.3665 | 0.6356 | 0.8922 | |
| COILPLM (Size)=BERT_base, Co-Training=false2022.06 | 0.355 | — | — | — | — | — | |
| RetroMAERep type=dense, Pre-training=true, Distillation=false, Target Corpus=false, Supervised=true2023.02 | 0.354 | 0.975 | — | — | — | — | |
| TAS-BRetriever Type=Dense Retriever2024.11 | 0.347 | 0.978 | — | — | — | — | |
| ContrieverRep type=dense, Pre-training=true, Distillation=false, Target Corpus=false, Supervised=true2023.02 | 0.341 | 0.979 | — | — | — | — | |
| ME-BERTPLM (Size)=BERT_large, Co-Training=false2022.06 | 0.338 | — | — | — | — | — | |
| CondenserFine-tuning strategy=DPR2022.05 | 0.3357 | 0.9683 | — | 0.3471 | 0.6082 | 0.877 | |
| ANCERetriever Type=Dense Retriever2024.11 | 0.33 | 0.959 | — | — | — | — | |
| ANCEInput Representation=FirstP2020.07 | 0.33 | 0.959 | — | — | — | — | |
| ANCEPLM (Size)=RoBERTa_base, Co-Training=false2022.06 | 0.33 | — | — | — | — | — | |
| SEEDFine-tuning strategy=DPR2022.05 | 0.3264 | 0.9539 | — | 0.3374 | 0.5913 | 0.8535 | |
| SPLADE-docRetriever Type=Inference-free Sparse Retriever2024.11 | 0.322 | 0.946 | — | — | — | — | |
| DiffCSEFine-tuning strategy=DPR2022.05 | 0.3202 | 0.9607 | — | 0.3311 | 0.5832 | 0.8561 | |
| SimCSEFine-tuning strategy=DPR2022.05 | 0.3193 | 0.9699 | — | 0.3307 | 0.5907 | 0.8653 | |
| LaPraDoRFine-tuning strategy=DPR2022.05 | 0.3191 | 0.9602 | — | 0.3307 | 0.5833 | 0.8537 | |
| DeBERTaFine-tuning strategy=DPR2022.05 | 0.3186 | 0.9654 | — | 0.3304 | 0.5824 | 0.8625 | |
| BERTFine-tuning strategy=DPR2022.05 | 0.317 | 0.9598 | — | 0.3278 | 0.5801 | 0.857 | |
| DPR-PAQPLM (Size)=BERT_base, Co-Training=false2022.06 | 0.314 | — | — | — | — | — | |
| ROBERTaFine-tuning strategy=DPR2022.05 | 0.3136 | 0.9579 | — | 0.3258 | 0.5638 | 0.8478 | |
| DPR (BM25 + Rand Neg)2020.07 | 0.311 | 0.952 | — | — | — | — | |
| DPR-PAQPLM (Size)=RoBERTa_base, Co-Training=false2022.06 | 0.311 | — | — | — | — | — | |
| BM25 -> BM25 + Rand2020.07 | 0.306 | 0.939 | — | — | — | — | |
| LSEPairObjective=LSEPair2026.02 | 0.3057 | — | — | — | — | 0.831 | |
| Rand1LHObjective=Rand1LH2026.02 | 0.3044 | — | — | — | — | 0.8202 | |
| SingleLHObjective=SingleLH2026.02 | 0.2991 | — | — | — | — | 0.8154 | |
| BM25 Neg2020.07 | 0.299 | 0.928 | — | — | — | — | |
| SumMargLHObjective=SumMargLH2026.02 | 0.2979 | — | — | — | — | 0.8076 | |
| Ensemble of 8 CKNRMConfiguration=Ensemble, Model Count=82019.03 | 0.29 | — | — | — | — | — | |
| JointLHObjective=JointLH2026.02 | 0.2822 | — | — | — | — | 0.8309 | |
| BM25 -> Rand2020.07 | 0.28 | 0.948 | — | — | — | — | |
| SparTermRetriever Type=Sparse Retriever2024.11 | 0.279 | 0.925 | — | — | — | — | |
| BM25 -> NCE Neg2020.07 | 0.279 | 0.942 | — | — | — | — | |
| IRNetDescription=proprietary deep neural model2019.03 | 0.278 | — | — | — | — | — | |
| doc2query-T5Retriever Type=Inference-free Sparse Retriever2024.11 | 0.277 | 0.947 | — | — | — | — | |
| docTTTTTqueryPLM (Size)=T5_base, Co-Training=false2022.06 | 0.277 | — | 0.756 | — | — | — | |
| Rand Neg2020.07 | 0.261 | 0.949 | — | — | — | — | |
| NCE Neg2020.07 | 0.256 | 0.943 | — | — | — | — | |
| Duet v2Configuration=Ensemble, Model Count=82019.03 | 0.252 | — | — | — | — | — | |
| Single CKNRMConfiguration=Single model2019.03 | 0.247 | — | — | — | — | — | |
| DeepCTRetriever Type=Inference-free Sparse Retriever2024.11 | 0.243 | 0.913 | — | — | — | — | |
| Duet v2Configuration=Single model2019.03 | 0.243 | — | — | — | — | — | |
| Best DeepCT2020.07 | 0.243 | — | — | — | — | — | |
| DeepCTPLM (Size)=BERT_base, Co-Training=false2022.06 | 0.243 | — | 0.69 | — | — | — | |
| BM252020.07 | 0.24 | 0.814 | — | — | — | — | |
| Best TREC Trad Retrieval2020.07 | 0.24 | — | — | — | — | — | |
| doc2queryPLM (Size)=BERT_base, Co-Training=false2022.06 | 0.215 | — | 0.644 | — | — | — | |
| Duet v2Configuration=Single model, Modification=w/o MLP to combine local and distributed scores2019.03 | 0.208 | — | — | — | — | — | |
| BM25Co-Training=false2022.06 | 0.187 | — | 0.592 | — | — | — | |
| BM25Retriever Type=Inference-free Sparse Retriever2024.11 | 0.184 | 0.853 | — | — | — | — | |
| Duet v2Configuration=Single model, Modification=w/ Tanh non-linearity (instead of ReLU)2019.03 | 0.179 | — | — | — | — | — | |
| BM252019.03 | 0.165 | — | — | — | — | — | |
| Duet v2Configuration=Single model, Modification=w/o IDF weighting for interaction matrix2019.03 | 0.163 | — | — | — | — | — |