Pubmed
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
PubMed (evaluation nodes)
75.83Accuracy
18
Pubmed (high-label)
89.87Accuracy
18
Pubmed low-label
0.777Accuracy
18
PubMed Claude3 Opus
62.7TPR @ FPR 1%
18
PubMed GPT4 Turbo
67.3TPR at FPR 1%
18
PubMed GPT4
70TPR at FPR 1%
18
PubMed GPT3.5 Turbo
63.3TPR @ FPR 1%
18
PubMed Attributed
0.99AUC
18
Pubmed
80.5Accuracy (Clean)
18
PubMed δ=3.5 (test)
81.3Test F1 Score
18
Pubmed Poisoned
84.71Accuracy
17
PubMed synthetic data IID setting
81.56Macro Accuracy
17
PubMed synthetic data IID setting
76.88Accuracy (C)
17
PubMed synthetic data IID setting
72.04Accuracy (A)
17
PubMed Synthetic GPT-2-large generated (test)
74.96Accuracy
17
PubMed Synthetic GPT-2-large generated (test)
87.88Accuracy
17
PUBMED δ = 3.5 (test)
96.3ROC AUC
17
Pubmed 20 labels per class
77.4Accuracy
17
Pubmed 5 labels per class
67.6Accuracy
17
PubMed 205w 1,000 samples (test)
0.422ROUGE-1 F1
16
PubMed
42.2ROUGE-1 F1
16
Pubmed 50/25/25 (test)
89.7Test Accuracy
16
Pubmed ID
98.25Accuracy
16
Pubmed (val)
36.5Perplexity
16
Pubmed ADD-0.75 (test)
0.761Accuracy
16