Toxicity detection on CivilComments-WILDS (test)
92.7Average AccuracyERM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ERMGroup annotated in train?=false, Group annotated in val?=false2023.05 | 92.7 | — | 51.6 | |
| ERMGroup annotated in train?=false, Group annotated in val?=true2023.05 | 92.6 | — | 57.4 | |
| ERM (ours)Backbone=BERT2021.04 | 92.5 | 58.1 | — | |
| CVaR DROGroup annotated in train?=false, Group annotated in val?=true2023.05 | 92.5 | — | 60.5 | |
| LfFGroup annotated in train?=false, Group annotated in val?=true2023.05 | 92.5 | — | 58.8 | |
| ERMBackbone=BERT2021.04 | 92.2 | 58 | — | |
| JTTGroup annotated in train?=false, Group annotated in val?=false2023.05 | 92.1 | — | 61.5 | |
| LfFGroup annotated in train?=false, Group annotated in val?=false2023.05 | 92 | — | 55.9 | |
| CVaR DROGroup annotated in train?=false, Group annotated in val?=false2023.05 | 91.9 | — | 56.5 | |
| FishBackbone=BERT2021.04 | 91.4 | 74.2 | — | |
| EIILGroup annotated in train?=false, Group annotated in val?=false2023.05 | 91.2 | — | 63.8 | |
| JTTGroup annotated in train?=false, Group annotated in val?=true2023.05 | 91.1 | — | 69.3 | |
| EIILGroup annotated in train?=false, Group annotated in val?=true2023.05 | 90.5 | — | 67 | |
| Group DROBackbone=BERT2021.04 | 89.4 | 70.4 | — | |
| SSAGroup annotated in train?=false, Group annotated in val?=false2023.05 | 89.1 | — | 69.5 | |
| ReweightedBackbone=BERT2021.04 | 88.9 | 67.3 | — | |
| Q-DiversityGroup annotated in train?=false, Group annotated in val?=false2023.05 | 88.7 | — | 73.5 | |
| SSAGroup annotated in train?=false, Group annotated in val?=true2023.05 | 88.2 | — | 69.9 | |
| Oracle DROGroup annotated in train?=true, Group annotated in val?=true2023.05 | 87.7 | — | 69.1 |