BeaverTails
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
BeaverTails (test)
7.9Harmful Score
110
BeaverTails bottom 30% uncertainty slice (test)
85.1AUROC
70
BeaverTails
84.6Accuracy
63
BeaverTails
80.1Accuracy (%)
63
BeaverTails HarmfulQA (1k and 10k samples)
0Avg Harmfulness Score
63
BeaverTails
87.3F1 Score
60
BeaverTails
89.9F1 Score
59
BeaverTails (test)
28.4Harmful Score
52
BeaverTails
8.8ASR
44
BeaverTails (test)
1Harmfulness Score
44
BeaverTails
0Harmful Response Probability
42
BeaverTails V Text-Image Response
84.8F1 Score
39
Beavertails
0.86F1 Score
35
Beavertails
58.4Helpful Score
33
Beavertails
1.2Violation Rate
32
BeaverTails V
93.37Safety Score
27
Beavertails
56.7Attack Success Rate
24
BeaverTails
0Attack Success Rate
24
BeaverTails (test)
59.9Value Alignment Score
24
BeaverTails (test)
94AUC
24
BeaverTails-V
2.9Safety Score
22
Beavertails-V (test)
86.93Helpfulness Score
20
BeaverTails Evaluation
1.24Harmful Score (HS)
20
BeaverTails
0.853AUROC
20
BeaverTails
9.88Ssafe Score
18