PubMedQA
Benchmarks
TaskDatasetSOTA resultTrendResults
PubMedQA (test)
82.4Accuracy
170
PubMedQA
83.6Accuracy
145
PubMedQA
81.4Accuracy
122
PubMedQA
82.8Accuracy
65
PubMedQA
79.23Accuracy
48
PubMedQA PQA-L (test)
87.08Accuracy
45
PubMedQA
83.67Accuracy
40
PubMedQA
77Attack Accuracy
40
PubmedQA
88F1 Score
36
PubMedQA (test)
76.03Accuracy
34
PubMedQA
86Pass@1
32
PubMedQA labeled (0.8:0.2)
76.8Accuracy
28
PubMedQA
95.63Factual Accuracy (FA)
28
PubMedQA MdQ
0PPL Change (%) vs Baseline
24
PubMedQA
79.82EM
18
PubMedQA long-context (PQA-L)
61.1Macro-F1
17
PubMedQA
14ASR (500)
16
PubMedQA
89.8Recall@1
15
PubMedQA
63.62Accuracy
15
PubMedQA
115.78Context Influence
15
PubMedQA
82.2Accuracy
15
PubMedQA
0.372PRR (ROUGE-L)
14
PubMedQA (out-of-domain)
11.7ROUGE-L
14
PubMedQA
68.32Accuracy
13
PubMedQA
182.24Throughput (tokens/s)
12