QASPER
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Qasper
1.66SR
12
Qasper
0.005Latency (s)
11
Qasper (test)
46.18F1 Score
11
QASPER
0.44Kendall's Tau
11
QASPER
72.73F1 Score
10
Qasper
54.57Extract F1
10
Qasper yes/no question answering
0.102AOPC@10
10
Qasper (val)
28.8F1
10
QASPER Multi-Document 4
76.2Accuracy
9
QASPER
90.68Refusal F1
8
QASPER n=250
37.3Ans-F1
8
QASPER n=250
56.8Sufficiency @5
8
QASPER Extractive (test)
53.3F1
8
QASPER Extractive (dev)
29.6F1
8
Qasper
13.91Accuracy
7
QASPER
15EM
7
QASPER
43Precision
6
QASPER (test)
0.44Mean Kendall Tau Correlation
6
QASPER Random context
71.7ROC AUC
5
QASPER No context
77.9ROC AUC
5
QASPER
28.5SECE Score
5
QASPER
60.9Accuracy
5
QASPER
78.2ROC AUC
5
QASPER
4Relative Evidence Count
4
QASPER NLP (test)
0.255ROUGE-L
4