CoQA
Benchmarks
TaskDatasetSOTA resultTrendResults
CoQA
90.07AUROC
134
CoQA
0.858Mean AUROC
107
CoQA
74.7ROC-AUC
66
CoQA
76.31CACC
64
CoQA
0.857AUROC
58
CoQA
0.423PRR
44
CoQA
77.5AUCs
42
CoQA (test)
77.3AUROC
42
CoQA alpha = 0.25 (test)
0.235Empirical Error Rate (EER)
40
CoQA alpha = 0.25 (filtering stage)
23.47EER
40
CoQA
91.74AUROC
39
CoQA
65.5Accuracy
35
CoQA
75.9Accuracy
35
COQA zero-shot (test)
70.85Exact Match (EM)
32
CoQA
76F1 Score
28
CoQA
71.4Exact Match
24
CoQA
40.7PRR
22
CoQA
94.61Accuracy
22
COQA
28.27Factual Accuracy
21
CoQA
0.98AUROC
20
CoQA
80.6PRR
20
CoQA
89.01AUPRC
20
CoQA official (test)
88.8Overall F1
17
CoQA (IID)
100Recall
16
CoQA (NIID-1)
100Recall
16