QASPER
Benchmarks
TaskDatasetSOTA resultTrendResults
QASPER (test)
61.5F1 Score (Match)
132
qasper
36.9F1 Score
61
Qasper
40.8Accuracy
44
Qasper
50.3F1 Score
44
Qasper
61.3Accuracy
37
QASPER
15.35Accuracy
35
QASPER (test)
30.27F1 Score
30
QASPER Long-doc
52.76Gen-F1
26
Qasper
44.79Score
22
QASPER 1200:251 (test)
28.92Answerable EM
20
Qasper
84Precision
18
Qasper
65EM Score
18
Qasper 128K context
39F1 Score
18
Qasper
83.09F1
17
Qasper
11.5Exact Match (EM)
16
QASPER
0.722AUROC
16
Qasper
0.368F1 Score
16
QASPER
38.8Rouge-L
15
Qasper
67.3Recall
15
QASPER NLP
87R@10
14
Qasper
30ASR Score
14
QASPER
100Recall@8
12
QASPER
233.2TTFT (ms)
12
QASPER
0.53Peak GPU Memory (GB)
12
QASPER (test)
36F1 Score
12