Group Robustness on CivilComments-WILDS (test)
67WG AccuracyIRM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| IRMGroup Info (Train/Val)=✓/★, Known Attribute=Black2026.02 | 67 | 90 | — | |
| LEIAGroup Info (Train/Val)=✗/★, Known Attribute=LGBTQ2026.02 | 66.8 | 90.1 | — | |
| Group DROGroup Info (Train/Val)=✓/★, Known Attribute=Black2026.02 | 66.7 | 89.5 | — | |
| DFRGroup Info (Train/Val)=✗/✓✓, Known Attribute=Black2026.02 | 66.5 | 90.6 | — | |
| LEIAGroup Info (Train/Val)=✗/★, Known Attribute=Black2026.02 | 66.1 | 90.2 | — | |
| LEIAGroup Info (Train/Val)=✗/★, Known Attribute=Muslim2026.02 | 65.2 | 89.5 | — | |
| IRMGroup Info (Train/Val)=✓/★, Known Attribute=LGBTQ2026.02 | 64.4 | 90.3 | — | |
| DFRGroup Info (Train/Val)=✗/✓✓, Known Attribute=Muslim2026.02 | 64.1 | 89.2 | — | |
| DFRGroup Info (Train/Val)=✗/✓✓, Known Attribute=LGBTQ2026.02 | 64 | 90.1 | — | |
| IRMGroup Info (Train/Val)=✓/★, Known Attribute=Muslim2026.02 | 63.6 | 89.3 | — | |
| ReWeightCRTGroup Info (Train/Val)=✗/★, Known Attribute=LGBTQ2026.02 | 63.4 | 90.7 | — | |
| Group DROGroup Info (Train/Val)=✓/★, Known Attribute=Muslim2026.02 | 63.3 | 89.4 | — | |
| CRTGroup Info (Train/Val)=✗/★, Known Attribute=LGBTQ2026.02 | 63.1 | 90.6 | — | |
| Group DROGroup Info (Train/Val)=✓/★, Known Attribute=LGBTQ2026.02 | 62.9 | 89.3 | — | |
| CRTGroup Info (Train/Val)=✗/★, Known Attribute=Black2026.02 | 62.8 | 90.4 | — | |
| ReWeightCRTGroup Info (Train/Val)=✗/★, Known Attribute=Muslim2026.02 | 62.1 | 90.4 | — | |
| ReWeightCRTGroup Info (Train/Val)=✗/★, Known Attribute=Black2026.02 | 61.9 | 90.7 | — | |
| LISAGroup Info (Train/Val)=✓/★, Known Attribute=Black2026.02 | 61.7 | 91.3 | — | |
| CRTGroup Info (Train/Val)=✗/★, Known Attribute=Muslim2026.02 | 61.5 | 90 | — | |
| LISAGroup Info (Train/Val)=✓/★, Known Attribute=Muslim2026.02 | 59.4 | 91.6 | — | |
| LISAGroup Info (Train/Val)=✓/★, Known Attribute=LGBTQ2026.02 | 59.2 | 91.7 | — | |
| JTTGroup Info (Train/Val)=✗/★, Known Attribute=Black2026.02 | 57.8 | 91.9 | — | |
| JTTGroup Info (Train/Val)=✗/★, Known Attribute=LGBTQ2026.02 | 57.7 | 91.9 | — | |
| JTTGroup Info (Train/Val)=✗/★, Known Attribute=Muslim2026.02 | 57.6 | 91.9 | — | |
| CVaRDROGroup Info (Train/Val)=✗/★, Known Attribute=Black2026.02 | 57.1 | 91.9 | — | |
| CVaRDROGroup Info (Train/Val)=✗/★, Known Attribute=Muslim2026.02 | 56.2 | 92 | — | |
| CVaRDROGroup Info (Train/Val)=✗/★, Known Attribute=LGBTQ2026.02 | 55.8 | 92.1 | — | |
| Contrast. AdapterBackbone=GPT-Neo 1.3B, Evaluation Protocol=Contrastive Adapter2022.07 | 50.1 | 54.2 | 1.1 | |
| DFR (Up)Backbone=GPT-Neo 1.3B, Evaluation Protocol=DFR (Up)2022.07 | 48.2 | 52.1 | 3.9 | |
| ERMGroup Info (Train/Val)=✗/★, Known Attribute=Black2026.02 | 48 | 92.3 | — | |
| ERMGroup Info (Train/Val)=✗/★, Known Attribute=LGBTQ2026.02 | 48 | 92.3 | — | |
| ERMGroup Info (Train/Val)=✗/★, Known Attribute=Muslim2026.02 | 48 | 92.3 | — | |
| DFR (Sub)Backbone=GPT-Neo 1.3B, Evaluation Protocol=DFR (Sub)2022.07 | 47.4 | 51.9 | 4.5 | |
| ERM LPBackbone=GPT-Neo 1.3B, Evaluation Protocol=ERM LP2022.07 | 46.7 | 51.2 | 4.5 | |
| WiSE-FTBackbone=GPT-Neo 1.3B, Evaluation Protocol=WiSE-FT2022.07 | 46.7 | 51.2 | 2.8 | |
| AFRGroup Info (Train/Val)=✗/★, Known Attribute=LGBTQ2026.02 | 38.4 | 92 | — | |
| AFRGroup Info (Train/Val)=✗/★, Known Attribute=Muslim2026.02 | 38.2 | 92 | — | |
| AFRGroup Info (Train/Val)=✗/★, Known Attribute=Black2026.02 | 38.1 | 92 | — | |
| ERM AdapterBackbone=GPT-Neo 1.3B, Evaluation Protocol=ERM Adapter2022.07 | 32.1 | 37.7 | 5.6 | |
| Zero-shotBackbone=GPT-Neo 1.3B, Evaluation Protocol=Zero-shot2022.07 | 16 | 74.8 | 58.8 |