List your company
ResearchFormerly WizWand

Open-domain question answering on NaturalQ-Open (test)

60.4EMATLAS
22.12832.0644251.936Sep 11, 2019Mar 5, 2020Aug 29, 2020Feb 21, 2021Aug 17, 2021Feb 9, 2022Aug 5, 2022
Updated 8mo ago

Evaluation results

MethodLinks
ATLASSetting=Full2022.0860.4
R2-D2Setting=Full2022.0855.9
FiD-KDSetting=Full2022.0854.7
FiDSetting=Full2022.0851.4
IDRQA2020.0945.5
RETROSetting=Full2022.0845.5
ATLASSetting=64-shot2022.0842.4
Dense Passage Retriever2020.0941.5
REALMArchitectures=Dense Retr.+Transformer, Pre-training=REALM, Pre-training Corpus=X = CC-News, Z = Wikipedia, # params=330m2020.0240.4
PaLMSetting=64-shot2022.0839.6
REALMArchitectures=Dense Retr.+Transformer, Pre-training=REALM, Pre-training Corpus=X = Wikipedia, Z = Wikipedia, # params=330m2020.0239.2
SPARTARanker Backbone=bert-base-uncased, Reader Backbone=SpanBERT2020.0937.5
ChinchillaSetting=64-shot2022.0835.5
T5 (11b)Architectures=Transformer Seq2Seq, Pre-training=T5 (Multitask), # params=11318m2020.0234.5
ORQA2019.1133.3
ORQAArchitectures=Dense Retr. +Transformer, Pre-training=ICT+BERT, # params=330m2020.0233.3
ORQA2020.0933.3
ORQA2020.0933.3
Recurrent Graph-based Retrieval2019.1132.6
PathRetrieverArchitectures=PathRetriever+Transformer, Pre-training=MLM, # params=110m2020.0232.6
Graph-retriever2020.0932.6
GraphRetrieverArchitectures=GraphRetriever+Transformer, Pre-training=BERT, # params=110m2020.0231.8
Recurrent Retriever2020.0931.7
GPT-3Setting=64-shot2022.0829.9
T5 (large)Architectures=Transformer Seq2Seq, Pre-training=T5 (Multitask), # params=738m2020.0229.8
GopherSetting=64-shot2022.0828.2
Discrete Hard EM2019.0928.1
Hard EM2019.1128.1
HardEMArchitectures=Sparse Retr.+Transformer, Pre-training=BERT, # params=110m2020.0228.1
Hard EM2020.0928.1
T5 (base)Architectures=Transformer Seq2Seq, Pre-training=T5 (Multitask), # params=223m2020.0227
Lee et al. (2019)2019.0926.5
BERT + BM 252019.1126.5
BERT-BaselineArchitectures=Sparse Retr.+Transformer, Pre-training=BERT, # params=110m2020.0226.5
BERT + BM252020.0926.5
MML2019.0925.8
First Only2019.0923.6