Information Retrieval on NaturalQuestions (NQ) (test)
82.24Top-20 AccTeacher
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TeacherLatency (ms)=17.152, Embedding Dimension=7682022.05 | 82.24 | 87.87 | — | — | |
| ANCELatency (ms)=6.159, Embedding Dimension=2562022.05 | 81.88 | 87.65 | — | — | |
| ANCELatency (ms)=3.419, Embedding Dimension=1282022.05 | 80.69 | 87.09 | — | — | |
| CELatency (ms)=7.344, Embedding Dimension=2562022.05 | 80.66 | 87.26 | — | — | |
| ConAELatency (ms)=3.942, Embedding Dimension=1282022.05 | 80.64 | 86.87 | — | — | |
| ConAELatency (ms)=7.158, Embedding Dimension=2562022.05 | 80.53 | 87.23 | — | — | |
| PCALatency (ms)=6.296, Embedding Dimension=2562022.05 | 80.42 | 87.15 | — | — | |
| DPR-supervised + C-MORESettings=Supervised, Training Data=Supervised Data + Wikipedia2022.03 | 80.3 | 86.7 | — | — | |
| CELatency (ms)=4.53, Embedding Dimension=1282022.05 | 79.34 | 86.68 | — | — | |
| PCALatency (ms)=3.525, Embedding Dimension=1282022.05 | 78.75 | 86.2 | — | — | |
| DPR-supervisedSettings=Supervised, Training Data=Supervised Data2022.03 | 78.4 | 85.4 | — | — | |
| ANCELatency (ms)=3.041, Embedding Dimension=642022.05 | 76.87 | 84.74 | — | — | |
| ConAELatency (ms)=3.087, Embedding Dimension=642022.05 | 76.04 | 84.6 | — | — | |
| CELatency (ms)=3.046, Embedding Dimension=642022.05 | 74.04 | 83.41 | — | — | |
| DPR-TQA + C-MORESettings=Domain Adaptation, Source Training Data=TriviaQA + Wikipedia2022.03 | 71 | 81.7 | — | — | |
| DPR-TQASettings=Domain Adaptation, Source Training Data=TriviaQA2022.03 | 69.2 | 80.3 | — | — | |
| DPR-WebQ + C-MORESettings=Domain Adaptation, Source Training Data=WebQ + Wikipedia2022.03 | 67.3 | 79.2 | — | — | |
| PCALatency (ms)=2.627, Embedding Dimension=642022.05 | 66.98 | 79.28 | — | — | |
| C-MORESettings=Unsupervised, Training Data=Wikipedia2022.03 | 61.9 | 75.8 | — | — | |
| REALMSettings=Unsupervised, Training Data=Wikipedia2022.03 | 59.8 | 74.9 | — | — | |
| BM25Settings=Unsupervised, Training Data=None2022.03 | 59.1 | 73.7 | — | — | |
| DPR-WebQSettings=Domain Adaptation, Source Training Data=WebQ2022.03 | 56.1 | 70.7 | — | — | |
| ORQASettings=Unsupervised, Training Data=Wikipedia2022.03 | 50.6 | 66.8 | — | — | |
| BART-GST-A + FiD-readerreranker=BART-GST-A (Attention Projection), reader=FiD-reader2023.05 | — | — | 79.3 | 83.3 | |
| BART-GST-M + FiD-readerreranker=BART-GST-M (MLP Projection), reader=FiD-reader2023.05 | — | — | 80 | 83.7 | |
| BART-reranker + FiD-GST-Areranker=BART-reranker, reader=FiD-GST-A (Attention Projection)2023.05 | — | — | 78.6 | 83.3 | |
| w/o reranker + FiD-GST-Areranker=None, reader=FiD-GST-A (Attention Projection)2023.05 | — | — | 74.6 | 80.3 |