Classification on CelebA (test)
95.9Average AccuracyBase (ERM)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Base (ERM)Group Info (Train/Val)=No/No2024.03 | 95.9 | 41.1 | — | |
| JTTGroup Info (Train/Val)=✗/✗2026.02 | 95.9 | 48.3 | — | |
| LISAGroup Info (Train/Val)=✗/✗2026.02 | 95.4 | 57.8 | — | |
| LEIAGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 95.2 | 85 | — | |
| LEIAGroup Info (Train/Val)=✗/★2026.02 | 95.2 | 85 | — | |
| LEIAGroup Info (Train/Val)=✗/✗2026.02 | 95.2 | 82.8 | — | |
| CVaRDRO*Group Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 95.1 | 62.2 | — | |
| CVaRDROGroup Info (Train/Val)=✗/★2026.02 | 95.1 | 62.2 | — | |
| CVaRDROGroup Info (Train/Val)=✗/✗2026.02 | 95.1 | 60.2 | — | |
| ERM*Group Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 95 | 57.6 | — | |
| ERMGroup Info (Train/Val)=✗/★2026.02 | 95 | 57.6 | — | |
| ERMGroup Info (Train/Val)=✗/✗2026.02 | 95 | 57.6 | — | |
| ERMGroup-oblivious setting=true2022.09 | 94.9 | 47.8 | — | |
| ERM2024.11 | 94.9 | 47.7 | — | |
| IRM*Group Info (Train/Val)=✓/★, Group labels usage in training=used2026.02 | 94.7 | 63 | — | |
| GEORGE (gDRO)Group Info (Train/Val)=No/No2024.03 | 94.6 | 53.7 | — | |
| ReWeightCRT*Group Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 94.2 | 71.1 | — | |
| ReWeightCRTGroup Info (Train/Val)=✗/★2026.02 | 94.2 | 71.1 | — | |
| ReWeightCRTGroup Info (Train/Val)=✗/✗2026.02 | 94.2 | 70.7 | — | |
| CRT*Group Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 94.1 | 70.4 | — | |
| CRTGroup Info (Train/Val)=✗/★2026.02 | 94.1 | 70.4 | — | |
| CRTGroup Info (Train/Val)=✗/✗2026.02 | 94.1 | 69.6 | — | |
| DFRGroup Info (Train/Val)=✗/✗2026.02 | 93.6 | 73.7 | — | |
| Group DROGroup Info (Train/Val)=Yes/Yes2024.03 | 92.9 | 88.9 | — | |
| Group DROGroup Info (Train/Val)=✓/★, Group labels usage in training=used2026.02 | 92.9 | 88.9 | — | |
| BARACKGroup Info (Train/Val)=Yes/Yes2024.03 | 92.8 | 83.8 | — | |
| LISAGroup Info (Train/Val)=✓/★, Group labels usage in training=used2026.02 | 92.4 | 89.3 | — | |
| GICGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 91.9 | 89.4 | — | |
| GICGroup Info (Train/Val)=✗/★2026.02 | 91.9 | 89.4 | — | |
| DFR+ExMapGroup Info (Train/Val)=No/No2024.03 | 91.8 | 84.4 | — | |
| MaskTuneGroup Info (Train/Val)=No/No2024.03 | 91.3 | 78 | — | |
| AFRGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 91.3 | 82 | — | |
| DFRGroup Info (Train/Val)=✗/✓✓, Group labels usage in training=used2026.02 | 91.3 | 88.3 | — | |
| AFRGroup Info (Train/Val)=✗/★2026.02 | 91.3 | 82 | — | |
| DFRGroup Info (Train/Val)=Yes/No2024.03 | 91.1 | 86.9 | — | |
| DPR2024.11 | 90.7 | 88.9 | — | |
| UMIXGroup-oblivious setting=true2022.09 | 90.1 | 85.3 | — | |
| CNC2024.11 | 89.9 | 88.8 | — | |
| CnCGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 89.9 | 88.8 | — | |
| CnCGroup Info (Train/Val)=✗/★2026.02 | 89.9 | 88.8 | — | |
| DPEGroup Info (Train/Val)=✗/✗2026.02 | 89.8 | 81.9 | — | |
| CVaR-DOROGroup-oblivious setting=true2022.09 | 89.6 | 75.6 | — | |
| EIILGroup Info (Train/Val)=Yes/Yes2024.03 | 89.5 | 81.3 | — | |
| GroupDRO (with Tgt)Group label=✓2026.01 | 89.3 | — | — | |
| GEORGE (DFR)Group Info (Train/Val)=No/No2024.03 | 89.2 | 83.3 | — | |
| PGD2024.11 | 88.6 | — | — | |
| Focal LossGroup-oblivious setting=true2022.09 | 88.4 | 72.1 | — | |
| JTT2024.11 | 88.1 | 81.5 | — | |
| JTTGroup-oblivious setting=true2022.09 | 88 | 81.1 | — | |
| JTTGroup Info (Train/Val)=Yes/No2024.03 | 88 | 81.1 | — | |
| JTTGroup Info (Train/Val)=No/No2024.03 | 88 | 40.6 | — | |
| JTTGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 88 | 81.1 | — | |
| JTTGroup Info (Train/Val)=✗/★2026.02 | 88 | 81.1 | — | |
| chi^2-DROGroup-oblivious setting=true2022.09 | 87.7 | 78.4 | — | |
| DPEGroup Info (Train/Val)=✗/✓, Group labels usage in training=used2026.02 | 87.7 | 89.8 | — | |
| chi^2-DOROGroup-oblivious setting=true2022.09 | 87 | 75.6 | — | |
| CVaR-DROGroup-oblivious setting=true2022.09 | 86.8 | 76.9 | — | |
| LfFGroup Info (Train/Val)=Yes/No2024.03 | 85.1 | 77.2 | — | |
| LfFGroup Info (Train/Val)=No/No2024.03 | 85.1 | 24.4 | — | |
| Ours (ERM)Group label=×2026.01 | 85 | — | — | |
| SHASAMBackbone=ViT, Variant=w/ FLCMI, ViT2026.02 | 84 | — | 5.3 | |
| FairViTBackbone=ViT, Implementation=Our re-implementation2026.02 | 83.8 | — | 6.4 | |
| CVaR DROGroup Info (Train/Val)=Yes/No2024.03 | 82.5 | 64.4 | — | |
| CVaR DROGroup Info (Train/Val)=No/No2024.03 | 82.5 | 36.1 | — | |
| SHASAMBackbone=Standard, Variant=w/ FLCMI2026.02 | 81.3 | — | 5.5 | |
| SHASAMBackbone=Standard, Variant=w/ LogDetCMI2026.02 | 80.7 | — | 6.1 | |
| SupConBackbone=Standard2026.02 | 80.5 | — | 30.5 | |
| LNLBackbone=Standard2026.02 | 79.9 | — | 21.8 | |
| FSCLBackbone=ViT, Variant=w/ ViT2026.02 | 79.9 | — | 5.7 | |
| CEBackbone=Standard2026.02 | 79.6 | — | 27.8 | |
| FSCLBackbone=Standard, Implementation=Our re-implementation, Config=w/ group norm2026.02 | 79.1 | — | 6.5 | |
| MFDBackbone=Standard2026.02 | 78 | — | 7.4 | |
| GRLBackbone=Standard2026.02 | 77.2 | — | 24.9 | |
| FD-VAEBackbone=Standard2026.02 | 76.9 | — | 15.1 | |
| *Ours (ERM)Group label=×2026.01 | 76 | — | — | |
| ASC2026.02 | 75.9 | 64.4 | — | |
| DPSGD2026.02 | 67.8 | 16.7 | — | |
| aZB-weakweaker privacy for minority classes=true2026.02 | 67 | 42.8 | — | |
| aZB-prop2026.02 | 65.3 | 36.9 | — | |
| GroupDROGroup label=✓2026.01 | 63 | — | — | |
| PDEGroup label=✓2026.01 | 55 | — | — | |
| aZB2026.02 | 51.2 | 4.6 | — | |
| ERM (Src-only)Group label=×2026.01 | 35.5 | — | — | |
| ATDOCGroup label=×2026.01 | 31.8 | — | — | |
| CORALGroup label=×2026.01 | 31.7 | — | — | |
| ICONGroup label=×2026.01 | 31.1 | — | — | |
| MCDGroup label=×2026.01 | 30.7 | — | — | |
| DRSTGroup label=×2026.01 | 29.5 | — | — | |
| MK-MMDGroup label=×2026.01 | 27.7 | — | — | |
| CDANGroup label=×2026.01 | 24.6 | — | — | |
| STARGroup label=×2026.01 | 24.4 | — | — | |
| DANNGroup label=×2026.01 | 23.5 | — | — | |
| CB last-layer retrainingAnnotations Group=false, Annotations Class=true2023.09 | — | 73.7 | — | |
| Class-balanced ERMAnnotations Group=false, Annotations Class=false2023.09 | — | 67.2 | — | |
| CnCAnnotations Group=false, Annotations Class=true2023.09 | — | 88.8 | — | |
| DFRAnnotations Group=true, Annotations Class=true2023.09 | — | 89.4 | — | |
| DFR (our impl.)Annotations Group=true, Annotations Class=true2023.09 | — | 87 | — | |
| DivDisGroup Info (Train/Val)=No/No2024.03 | — | 55 | — | |
| ES disagreement SELFAnnotations Group=false, Annotations Class=false2023.09 | — | 83.9 | — | |
| Group DRO-ESAnnotations Group=true, Annotations Class=true2023.09 | — | 90.6 | — |