CivilComments
Benchmarks
TaskDatasetSOTA resultTrendResults
CivilComments sensitive attribute: MUSLIM (test)
59.9Balanced Accuracy
57
CivilComments (test)
92.2Average Accuracy
51
CivilComments
72.6Worst-Group Accuracy
23
CivilComments BERT (test)
0.57Oracle ECE
20
CivilComments (16 groups)
86.3Average Accuracy
20
CivilComments-WILDS (test)
92.7Average Accuracy
19
CivilComments HELM
65.81Balanced Acc
18
CivilComments
81Worst-Group Accuracy
17
CivilComments (test)
78.8WGA
14
CivilComments-WILDS (test)
92.34Accuracy
13
CivilComments (CC) (test)
79.66Worst-Group Accuracy
13
CivilComments WILDS
75.3Worst-Group Accuracy
11
CivilComments
92.6Average Accuracy
10
CivilComments (test)
65.4Macro F1-Score
8
CivilComments Hard subset - demographic identities
86.9Hard Accuracy
7
CivilComments WILDS
85.5Average Accuracy
6
CivilComments (val)
69.1Accuracy
6
CivilComments Wilds (test)
92.2Average Accuracy
6
CivilComments Wilds (val)
92.3Average Accuracy
6
CivilComments BERT (test)
1.08ECE (Oracle Estimate)
5
CivilComments controlled shortcut injection
57.2Accuracy
5
CivilComments
0.092ECE
4
CivilComments group-balanced (evaluation)
68Bias Reduction (%)
3
CivilComments (test)
85.76Stage 1 Score
2