Image Classification on Waterbirds (test)
97.1Worst-Group AccuracyDFR
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DFRBackbone=ViT-L (300M), Requires group labels=true, Training=Trained on Waterbirds2026.06 | 97.1 | — | — | — | — | 97.7 | — | — | — | — | — | — | — | |
| DFRBackbone=ViT-G (1.1B), Requires group labels=true, Training=Trained on Waterbirds2026.06 | 96.4 | — | — | — | — | 98.1 | — | — | — | — | — | — | — | |
| Center CropBackbone=ViT-G (1.1B), Image size=224x224, Pretrained=DINOv22026.06 | 94.9 | — | — | — | — | 98 | — | — | — | — | — | — | — | |
| A2 cross-modelBackbone=ViT-S x ViT-G, Seeds=52026.06 | 94.4 | — | — | — | — | 98.2 | — | — | — | — | — | — | — | |
| DFRBackbone=ViT-B (86M), Requires group labels=true, Training=Trained on Waterbirds2026.06 | 94 | — | — | — | — | 95.5 | — | — | — | — | — | — | — | |
| ES disagreement SELFAnnotations Group=false, Annotations Class=false2023.09 | 93 | — | — | — | — | — | — | — | — | — | — | — | — | |
| AutoBackSwapTraining setting=with minority groups, Backbone=ResNet-502026.06 | 93 | — | — | — | — | — | — | — | — | — | — | 95 | — | |
| DFRBias labels used for training (TR.)=false, Bias labels used for validation (VAL)=true2025.12 | 92.9 | — | — | — | — | — | — | 94.2 | — | — | — | — | — | |
| DFRGroup Info (Train/Val)=✗/✓✓, Group labels usage in training=used2026.02 | 92.9 | — | — | — | — | 94.2 | — | — | — | — | — | — | — | |
| DFRBias labels=val, Retraining=last layer, Backbone=ResNet-502026.05 | 92.9 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DFRBackbone=ResNet-50, Requires group labels=true, Training=Trained on Waterbirds2026.06 | 92.9 | — | — | — | — | 94.2 | — | — | — | — | — | — | — | |
| DFRBackbone=ViT-B/162025.10 | 92.9 | — | — | — | — | 94.2 | — | — | — | — | — | — | — | |
| Full onlyBackbone=ViT-G (1.1B), Pretrained=DINOv2, Image size=224x2242026.06 | 92.8 | — | — | — | — | 97.8 | — | — | — | — | — | — | — | |
| CB last-layer retrainingAnnotations Group=false, Annotations Class=true2023.09 | 92.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DFR+ExMapTrain Group Info=false, Val Group Info=false2024.03 | 92.6 | — | — | — | — | 96 | — | — | — | — | — | — | — | |
| DFR+ExMapGroup Info (Train/Val)=No/No2024.03 | 92.5 | — | — | — | — | 96 | — | — | — | — | — | — | — | |
| GRSSLgroup_info_train=used, group_info_val=used2026.07 | 92.5 | — | — | — | — | 95.4 | — | — | — | — | — | — | — | |
| DFR (our impl.)Annotations Group=true, Annotations Class=true2023.09 | 92.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DaCgroup_info_train=not used, group_info_val=used2026.07 | 92.4 | — | — | — | — | 95.3 | — | — | — | — | — | — | — | |
| DFRgroup_info_train=not used, group_info_val=used2026.07 | 92.3 | — | — | — | — | 93.3 | — | — | — | — | — | — | — | |
| AutoBackSwapTraining setting=without minority groups, Backbone=ResNet-502026.06 | 92.2 | — | — | — | — | — | — | — | — | — | — | 95.1 | — | |
| DFRGroup Info (Train/Val)=Yes/No2024.03 | 92.1 | — | — | — | — | 96.7 | — | — | — | — | — | — | — | |
| DFRTrain Group Info=false, Val Group Info=true2024.03 | 92.1 | — | — | — | — | 96.7 | — | — | — | — | — | — | — | |
| GEORGE (DFR)Group Info (Train/Val)=No/No2024.03 | 91.7 | — | — | — | — | 96.5 | — | — | — | — | — | — | — | |
| Group DROgroup_info_train=used, group_info_val=used2026.07 | 91.6 | — | — | — | — | 93.4 | — | — | — | — | — | — | — | |
| Group DROGroup Info (Train/Val)=Yes/Yes2024.03 | 91.4 | — | — | — | — | 93.5 | — | — | — | — | — | — | — | |
| GroupDROBias labels used for training (TR.)=true, Bias labels used for validation (VAL)=true2025.12 | 91.4 | — | — | — | — | — | — | 93.5 | — | — | — | — | — | |
| Group DROGroup Info (Train/Val)=✓/★, Group labels usage in training=used2026.02 | 91.4 | — | — | — | — | 93.5 | — | — | — | — | — | — | — | |
| Group DROBias labels=train+val, Retraining=full, Backbone=ResNet-502026.05 | 91.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Group DROBackbone=ResNet-50, Requires group labels=true, Training=Trained on Waterbirds2026.06 | 91.4 | — | — | — | — | 93.5 | — | — | — | — | — | — | — | |
| SCER2025.11 | 91.2 | — | — | — | — | 92.1 | — | — | — | — | — | — | — | |
| NSFBias labels used for training (TR.)=false, Bias labels used for validation (VAL)=false2025.12 | 91.12 | — | — | — | — | — | — | 95.65 | — | — | — | — | — | |
| DFRAnnotations Group=true, Annotations Class=true2023.09 | 91.1 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DPEGroup Info (Train/Val)=✗/✓, Group labels usage in training=used2026.02 | 91 | — | — | — | — | 92.5 | — | — | — | — | — | — | — | |
| DPEGroup Info (Train/Val)=✗/✗2026.02 | 91 | — | — | — | — | 92.5 | — | — | — | — | — | — | — | |
| DFR2025.11 | 91 | — | — | — | — | 92.3 | — | — | — | — | — | — | — | |
| DFR + A2Backbone=ViT-S (21M), Requires group labels=true, Training=Trained on Waterbirds2026.06 | 91 | — | — | — | — | 91.9 | — | — | — | — | — | — | — | |
| Group DRO-ESAnnotations Group=true, Annotations Class=true2023.09 | 90.7 | — | — | — | — | — | — | — | — | — | — | — | — | |
| DRO-B2TGT bias labels=false2023.01 | 90.7 | — | — | — | — | 92.1 | — | — | — | — | — | — | — | |
| LEIAGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 90.7 | — | — | — | — | 93.3 | — | — | — | — | — | — | — | |
| LEIAGroup Info (Train/Val)=✗/★2026.02 | 90.7 | — | — | — | — | 93.3 | — | — | — | — | — | — | — | |
| DFRTraining setting=with minority groups, Backbone=ResNet-502026.06 | 90.7 | — | — | — | — | — | — | — | — | — | — | 94.3 | — | |
| TCSL-CSGroup Info Train=false, Coreset Selection Ratio (r)=0.22026.06 | 90.6 | — | — | — | — | 96.68 | — | — | — | — | — | — | — | |
| AFRBias labels used for training (TR.)=false, Bias labels used for validation (VAL)=false2025.12 | 90.4 | — | — | — | — | — | — | 94.2 | — | — | — | — | — | |
| AFRGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 90.4 | — | — | — | — | 94.2 | — | — | — | — | — | — | — | |
| AFRGroup Info (Train/Val)=✗/★2026.02 | 90.4 | — | — | — | — | 94.2 | — | — | — | — | — | — | — | |
| DFRBackbone=ViT-S (21M), Requires group labels=true, Training=Trained on Waterbirds2026.06 | 90.4 | — | — | — | — | 92.4 | — | — | — | — | — | — | — | |
| PDE2025.11 | 90.3 | — | — | — | — | 92.4 | — | — | — | — | — | — | — | |
| Full onlyBackbone=ViT-L (300M), Pretrained=DINOv2, Image size=518x5182026.06 | 90.3 | — | — | — | — | 97.7 | — | — | — | — | — | — | — | |
| LEIAGroup Info (Train/Val)=✗/✗2026.02 | 90.1 | — | — | — | — | 93.7 | — | — | — | — | — | — | — | |
| DROGT bias labels=true2023.01 | 89.9 | — | — | — | — | 91.5 | — | — | — | — | — | — | — | |
| Full onlyBackbone=ViT-B (86M), Pretrained=DINOv2, Image size=518x5182026.06 | 89.9 | — | — | — | — | 96.3 | — | — | — | — | — | — | — | |
| Attending on Attention (A2)Backbone=ViT-G (1.1B), Pretrained=DINOv2, Seeds=52026.06 | 89.9 | — | — | — | — | 95.8 | — | — | — | — | — | — | — | |
| BARACKGroup Info (Train/Val)=Yes/Yes2024.03 | 89.6 | — | — | — | — | 94.3 | — | — | — | — | — | — | — | |
| EL2N (Top)Group Info Train=false, Coreset Selection Ratio (r)=0.22026.06 | 89.56 | — | — | — | — | 98.05 | — | — | — | — | — | — | — | |
| D2 (EL2N+ResNet)Group Info Train=false, Coreset Selection Ratio (r)=0.22026.06 | 89.56 | — | — | — | — | 98.26 | — | — | — | — | — | — | — | |
| Full onlyBackbone=ViT-L (300M), Pretrained=DINOv2, Image size=224x2242026.06 | 89.3 | — | — | — | — | 96.9 | — | — | — | — | — | — | — | |
| LISAGroup Info (Train/Val)=✓/★, Group labels usage in training=used2026.02 | 89.2 | — | — | — | — | 91.8 | — | — | — | — | — | — | — | |
| LISAgroup_info_train=used, group_info_val=used2026.07 | 89.2 | — | — | — | — | 91.8 | — | — | — | — | — | — | — | |
| DFRGroup Info (Train/Val)=✗/✗2026.02 | 89 | — | — | — | — | 92.2 | — | — | — | — | — | — | — | |
| SUBGms=0.73, mt=0.392022.11 | 88.87 | — | — | — | — | 93.48 | — | — | — | — | — | — | — | |
| ElRep2025.11 | 88.8 | — | — | — | — | 92.9 | — | — | — | — | — | — | — | |
| LISA2025.11 | 88.7 | — | — | — | — | 92.8 | — | — | — | — | — | — | — | |
| Attending on Attention (A2)Backbone=ViT-B (86M), Pretrained=DINOv2, Seeds=52026.06 | 88.6 | — | — | — | — | 96.1 | — | — | — | — | — | — | — | |
| CnCAnnotations Group=false, Annotations Class=true2023.09 | 88.5 | — | — | — | — | — | — | — | — | — | — | — | — | |
| CNCGT bias labels=false2023.01 | 88.5 | — | — | — | — | 90.9 | — | — | — | — | — | — | — | |
| CNCBias labels used for training (TR.)=false, Bias labels used for validation (VAL)=false2025.12 | 88.5 | — | — | — | — | — | — | 90.9 | — | — | — | — | — | |
| CnCGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 88.5 | — | — | — | — | 90.9 | — | — | — | — | — | — | — | |
| CnCGroup Info (Train/Val)=✗/★2026.02 | 88.5 | — | — | — | — | 90.9 | — | — | — | — | — | — | — | |
| LA/TTLSAms=0.73, mt=0.392022.11 | 88.38 | — | — | — | — | 95.23 | — | — | — | — | — | — | — | |
| DDBgroup_info_train=not used, group_info_val=used2026.07 | 88.37 | — | — | — | — | 94.2 | — | — | — | — | — | — | — | |
| gDROms=0.73, mt=0.392022.11 | 87.98 | — | — | — | — | 93.06 | — | — | — | — | — | — | — | |
| RGbalGroup Info Train=true, Coreset Selection Ratio (r)=0.22026.06 | 87.49 | — | — | — | — | 96.98 | — | — | — | — | — | — | — | |
| EIILGroup Info (Train/Val)=Yes/Yes2024.03 | 87.3 | — | — | — | — | 93.1 | — | — | — | — | — | — | — | |
| JTT+ExMapTrain Group Info=false, Val Group Info=false2024.03 | 86.9 | — | — | — | — | 90 | — | — | — | — | — | — | — | |
| ReWeight2025.11 | 86.9 | — | — | — | — | 91.8 | — | — | — | — | — | — | — | |
| JTTGroup Info (Train/Val)=Yes/No2024.03 | 86.7 | — | — | — | — | 93.3 | — | — | — | — | — | — | — | |
| JTTTrain Group Info=false, Val Group Info=true2024.03 | 86.7 | — | — | — | — | 93.3 | — | — | — | — | — | — | — | |
| JTTBias labels used for training (TR.)=false, Bias labels used for validation (VAL)=false2025.12 | 86.7 | — | — | — | — | — | — | 93.3 | — | — | — | — | — | |
| JTTGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 86.7 | — | — | — | — | 93.3 | — | — | — | — | — | — | — | |
| JTTGroup Info (Train/Val)=✗/★2026.02 | 86.7 | — | — | — | — | 93.3 | — | — | — | — | — | — | — | |
| JTTBias labels=val, Retraining=full, Backbone=ResNet-502026.05 | 86.7 | — | — | — | — | — | — | — | — | — | — | — | — | |
| JTTBackbone=ResNet-50, Training=Trained on Waterbirds2026.06 | 86.7 | — | — | — | — | 93.3 | — | — | — | — | — | — | — | |
| MaskTuneGroup Info (Train/Val)=No/No2024.03 | 86.4 | — | — | — | — | 93 | — | — | — | — | — | — | — | |
| MTBias labels used for training (TR.)=false, Bias labels used for validation (VAL)=false2025.12 | 86.4 | — | — | — | — | — | — | 93 | — | — | — | — | — | |
| GICGroup Info (Train/Val)=✗/★, Group labels usage in training=not used2026.02 | 86.3 | — | — | — | — | 89.6 | — | — | — | — | — | — | — | |
| GICGroup Info (Train/Val)=✗/★2026.02 | 86.3 | — | — | — | — | 89.6 | — | — | — | — | — | — | — | |
| CBLoss2025.11 | 86.2 | — | — | — | — | 91.3 | — | — | — | — | — | — | — | |
| uLABias labels used for training (TR.)=false, Bias labels used for validation (VAL)=true2025.12 | 86.1 | — | — | — | — | — | — | 91.5 | — | — | — | — | — | |
| D2 (EL2N+CLIP)Group Info Train=false, Coreset Selection Ratio (r)=0.22026.06 | 86.09 | — | — | — | — | 98.22 | — | — | — | — | — | — | — | |
| GroupDROBackbone=ViT-B/162025.10 | 86 | — | — | — | — | 96 | — | — | — | — | — | — | — | |
| ERMms=0.73, mt=0.392022.11 | 85.78 | — | — | — | — | 93.19 | — | — | — | — | — | — | — | |
| Chang et al.Training setting=with minority groups, Backbone=ResNet-502026.06 | 85.7 | — | — | — | — | — | — | — | — | — | — | 95.7 | — | |
| JTTAnnotations Group=false, Annotations Class=true2023.09 | 85.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Attending on Attention (A2)Backbone=ViT-L (300M), Pretrained=DINOv2, Seeds=52026.06 | 85.2 | — | — | — | — | 93.8 | — | — | — | — | — | — | — | |
| Full onlyBackbone=ViT-B (86M), Pretrained=DINOv2, Image size=224x2242026.06 | 84 | — | — | — | — | 94.4 | — | — | — | — | — | — | — | |
| MMD2025.11 | 83.9 | — | — | — | — | 93 | — | — | — | — | — | — | — | |
| SelfSup (Top)Group Info Train=false, Coreset Selection Ratio (r)=0.22026.06 | 83.85 | — | — | — | — | 97.8 | — | — | — | — | — | — | — | |
| D2 (SelfSup+ResNet)Group Info Train=false, Coreset Selection Ratio (r)=0.22026.06 | 83.84 | — | — | — | — | 96.97 | — | — | — | — | — | — | — | |
| JTTGT bias labels=false2023.01 | 83.8 | — | — | — | — | 89.3 | — | — | — | — | — | — | — |