Image Classification on Waterbirds
97.2WG AccuracyEarly mask (GT)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Early mask (GT)Arch.=ViT-B, K=1, Extra supervision=true, Ground Truth Masks=true2025.06 | 97.2 | 99.2 | — | — | — | |
| iFAMArch.=ViT-B, K=82025.06 | 97 | 99 | — | — | — | |
| DFRBackbone (attn x emb)=ViT-G (1.1B), Evaluation Protocol=Deep Feature Reweighting (requires group labels)2026.06 | 96.4 | — | — | — | — | |
| iFAM (K=8)Backbone (attn x emb)=ViT-B (86M), Evaluation Protocol=End-to-end attention learning2026.06 | 96.1 | — | — | — | — | |
| iFAM (K=4)Backbone (attn x emb)=ViT-B (86M), Evaluation Protocol=End-to-end attention learning2026.06 | 96 | — | — | — | — | |
| DinoV2 (Fine-tuned)Arch.=ViT-B, Backbone State=Fine-tuned2025.06 | 95.8 | 98.6 | — | — | — | |
| Early mask FOUNDArch.=ViT-B, K=12025.06 | 95.2 | 98.6 | — | — | — | |
| A²_LR cross-modelBackbone (attn x emb)=ViT-S × ViT-G, Evaluation Protocol=Attending on Attention (ours)2026.06 | 94.4 | — | — | — | — | |
| DinoV2 PCAArch.=ViT-B2025.06 | 94 | 97.4 | — | — | — | |
| DFRBackbone (attn x emb)=ViT-B (86M), Evaluation Protocol=Deep Feature Reweighting (requires group labels)2026.06 | 94 | — | — | — | — | |
| DinoV2 + AIMArch.=ViT-B, Backbone State=Frozen2025.06 | 93.9 | 97.6 | — | — | — | |
| HSFMGroup Info (train/val)=✗/✓2026.03 | 93.1 | 94 | — | — | — | |
| DDBGroup Info (train/val)=✗/✓2026.03 | 93 | 93.6 | — | — | — | |
| SELF**6Annot. Tr/Val=✗/✓2024.07 | 93 | — | — | — | — | |
| DFR2025.10 | 92.9 | 94.2 | — | — | — | |
| LR (full image)Backbone (attn x emb)=ViT-G (1.1B), Evaluation Protocol=Linear probe on full image (no crops)2026.06 | 92.8 | — | — | — | — | |
| ExMapuses training group labels=false, uses validation group labels=false, inference-time interpretability=false2026.06 | 92.5 | 96 | — | — | — | |
| DFRGroup Info (train/val)=✗/✓✓2026.03 | 92.3 | 93.3 | — | — | — | |
| DaCGroup Info (train/val)=✗/✓2026.03 | 92.3 | 95.3 | — | — | — | |
| DaC2025.10 | 92.3 | 95.3 | — | — | — | |
| DFRuses training group labels=false, uses validation group labels=true, inference-time interpretability=false2026.06 | 92.1 | 96.7 | — | — | — | |
| SPARE**Annot. Tr/Val=✗/✓2024.07 | 91.6 | 96.2 | — | — | — | |
| DFRAnnot. Tr/Val=✓/✓2024.07 | 91.5 | 93.7 | — | — | — | |
| Group DROGroup Info (train/val)=✓/✓2026.03 | 91.4 | 93.5 | — | — | — | |
| RCAPPrune Rate=90%, Time (minutes)=10.02026.06 | 91.21 | — | — | — | — | |
| SCERBackbone=ResNet-502025.11 | 91.2 | 92.1 | — | — | — | |
| DinoV2 + EPArch.=ViT-B, Backbone State=Frozen2025.06 | 91 | 96 | — | — | — | |
| DFR† + A²Backbone (attn x emb)=ViT-S × ViT-S, Evaluation Protocol=Deep Feature Reweighting (requires group labels), uses group labels=true2026.06 | 91 | — | — | — | — | |
| Group DROuses training group labels=true, uses validation group labels=true, inference-time interpretability=false2026.06 | 90.7 | 92.7 | — | — | — | |
| GroupDROGroup labels in train set=Yes2022.09 | 90.6 | 91.8 | — | — | — | |
| GroupDROGroup labels in train set?=Yes2022.09 | 90.6 | 91.8 | — | — | — | |
| GroupDROGroup labels in train set?=Yes2022.09 | 90.6 | 91.8 | — | — | — | |
| GroupDROArch.=R-502025.06 | 90.6 | 91.8 | — | — | — | |
| SliceLensRepair Protocol=Discriminative Fine-tuning for Repair (DFR) [18]2025.12 | 90.4 | — | — | — | — | |
| DFRBackbone (attn x emb)=ViT-S (21M), Evaluation Protocol=Deep Feature Reweighting (requires group labels)2026.06 | 90.4 | — | — | — | — | |
| AFR2025.10 | 90.4 | 94.2 | — | — | — | |
| PDEBackbone=ResNet-502025.11 | 90.3 | 92.4 | — | — | — | |
| UMIXGroup labels in train set=No2022.09 | 90 | 93 | — | — | — | |
| UMIXGroup labels in train set?=No2022.09 | 90 | 93 | — | — | — | |
| UMIXGroup labels in train set?=No2022.09 | 90 | 93 | — | — | — | |
| CnC**Annot. Tr/Val=✗/✓2024.07 | 89.9 | 92 | — | — | — | |
| DominoRepair Protocol=Discriminative Fine-tuning for Repair (DFR) [18]2025.12 | 89.8 | — | — | — | — | |
| DeDiER**Annot. Tr/Val=✗/✓2024.07 | 89.8 | 92.1 | — | — | — | |
| LadderRepair Protocol=Native ensembling procedure2025.12 | 89.6 | — | — | — | — | |
| LISAGroup labels in train set=Yes2022.09 | 89.2 | 91.8 | — | — | — | |
| LISAGroup labels in train set?=Yes2022.09 | 89.2 | 91.8 | — | — | — | |
| LISAGroup labels in train set?=Yes2022.09 | 89.2 | 91.8 | — | — | — | |
| LISAGroup Info (train/val)=✓/✓2026.03 | 89.2 | 91.8 | — | — | — | |
| FACTSRepair Protocol=Discriminative Fine-tuning for Repair (DFR) [18]2025.12 | 88.9 | — | — | — | — | |
| ElRepBackbone=ResNet-502025.11 | 88.8 | 92.9 | — | — | — | |
| LISABackbone=ResNet-502025.11 | 88.7 | 92.8 | — | — | — | |
| DISCArch.=R-502025.06 | 88.7 | 93.8 | — | — | — | |
| TCSL-CSGroup Info Train=false, Selection Ratio (r)=0.4, Retraining Method=CB ERM2026.06 | 88.63 | 92.55 | — | — | — | |
| DinoV2 (Frozen)Arch.=ViT-B, Backbone State=Frozen2025.06 | 88.5 | 95.9 | — | — | — | |
| CnC2025.10 | 88.5 | 90.9 | — | — | — | |
| CnCuses training group labels=false, uses validation group labels=true, inference-time interpretability=false2026.06 | 88.5 | 90.9 | — | — | — | |
| NCTuses training group labels=false, uses validation group labels=false, inference-time interpretability=true2026.06 | 88 | 92.8 | — | — | — | |
| EIILuses training group labels=false, uses validation group labels=true, inference-time interpretability=false2026.06 | 87.3 | 93.1 | — | — | — | |
| GroupDROAnnot. Tr/Val=✓/✓2024.07 | 87 | 90.3 | — | — | — | |
| Contrastive AdapterBackbone=CLIP ViT-L/142022.07 | 86.9 | 96.2 | 9.3 | — | — | |
| JTTGroup Info (train/val)=✗/✓2026.03 | 86.7 | 93.3 | — | — | — | |
| JTT2025.10 | 86.7 | 93.6 | — | — | — | |
| JTTuses training group labels=false, uses validation group labels=true, inference-time interpretability=false2026.06 | 86.7 | 93.3 | — | — | — | |
| RGbalGroup Info Train=true, Selection Ratio (r)=0.4, Retraining Method=CB ERM2026.06 | 86.55 | 97.97 | — | — | — | |
| MaskTuneArch.=R-502025.06 | 86.4 | 93 | — | — | — | |
| JTTSetting=group-oblivious2022.09 | 86 | 93.6 | — | — | — | |
| GroupDRO2025.10 | 86 | 96 | — | — | — | |
| Contrastive AdapterFoundation Model=CLOOB, Backbone=RN-50x42022.07 | 85.8 | 88.5 | 2.7 | — | — | |
| EL2N (Top)Group Info Train=false, Selection Ratio (r)=0.4, Retraining Method=CB ERM2026.06 | 85.25 | 98.39 | — | — | — | |
| CeGDROBackbone=ResNet-50, Pre-trained=ImageNet, Selection Criteria=Worst group accuracy2024.11 | 84.8 | 90.3 | — | — | — | |
| D2 (EL2N+ResNet)Group Info Train=false, Selection Ratio (r)=0.4, Retraining Method=CB ERM2026.06 | 84.79 | 98.35 | — | — | — | |
| CeGDRO - EFBackbone=ResNet-50, Pre-trained=ImageNet, Selection Criteria=Worst group accuracy2024.11 | 84.3 | 90 | — | — | — | |
| PDiscoFormerArch.=ViT-B, K=82025.06 | 84.3 | 94.2 | — | — | — | |
| Late mask (GT)Arch.=ViT-B, K=1, Extra supervision=true, Ground Truth Masks=true2025.06 | 84 | 95.7 | — | — | — | |
| Contrastive AdapterBackbone=CLIP RN-50, Evaluation Protocol=Contrastive Adaptation2022.07 | 83.7 | 89.4 | 5.7 | — | — | |
| Contrastive AdapterBackbone=CLIP ResNet-502022.07 | 83.7 | 89.4 | 5.7 | — | — | |
| SelfSup (Top)Group Info Train=false, Selection Ratio (r)=0.4, Retraining Method=CB ERM2026.06 | 83.64 | 98.22 | — | — | — | |
| RGbalGroup Info Train=true, Coreset fraction (r)=0.6, Retraining Method=CB ERM2026.06 | 83.54 | 97.91 | — | — | — | |
| TCSL-CSGroup Info Train=false, Coreset fraction (r)=0.6, Retraining Method=CB ERM2026.06 | 83.52 | 95.08 | — | — | — | |
| D2 (SelfSup+ResNet)Group Info Train=false, Selection Ratio (r)=0.4, Retraining Method=CB ERM2026.06 | 83.43 | 97.61 | — | — | — | |
| DATBackbone=CLIP (ViT-L/14), Zero-shot=true2026.06 | 83.33 | 89.57 | 6.42 | — | — | |
| Late mask FOUNDArch.=ViT-B, K=12025.06 | 83.3 | 95.3 | — | — | — | |
| Contrastive AdapterFoundation Model=CLIP, Backbone=ViT-B/162022.07 | 83.1 | 90.9 | 7.8 | — | — | |
| Contrastive AdapterFoundation Model=CLOOB, Backbone=RN-502022.07 | 83 | 86.8 | 3.8 | — | — | |
| PruSCAnnot. Tr/Val=✗/✗2024.07 | 82.5 | 89.4 | — | — | — | |
| D2 (SelfSup+ResNet)Group Info Train=false, Coreset fraction (r)=0.6, Retraining Method=CB ERM2026.06 | 82.5 | 97.99 | — | — | — | |
| Contrastive AdapterFoundation Model=CLIP, Backbone=RN-1012022.07 | 82 | 86 | 4 | — | — | |
| EL2N (Hist)Group Info Train=false, Coreset fraction (r)=0.6, Retraining Method=CB ERM2026.06 | 81.31 | 97.81 | — | — | — | |
| SelfSup (Hist)Group Info Train=false, Coreset fraction (r)=0.6, Retraining Method=CB ERM2026.06 | 81.09 | 97.77 | — | — | — | |
| D2 (EL2N+ResNet)Group Info Train=false, Coreset fraction (r)=0.6, Retraining Method=CB ERM2026.06 | 81 | 98.27 | — | — | — | |
| SelfSup (Top)Group Info Train=false, Coreset fraction (r)=0.6, Retraining Method=CB ERM2026.06 | 80.74 | 98.16 | — | — | — | |
| Contrastive AdapterFoundation Model=CLIP, Backbone=ViT-B/322022.07 | 80.7 | 84.2 | 3.5 | — | — | |
| ERMArch.=ViT-B2025.06 | 80.7 | 95 | — | — | — | |
| A²_LRBackbone (attn x emb)=ViT-S × ViT-S, Evaluation Protocol=Attending on Attention (ours)2026.06 | 80.4 | — | — | — | — | |
| CORALGroup labels in train set=Yes2022.09 | 79.8 | 90.3 | — | — | — | |
| CORALGroup labels in train set?=Yes2022.09 | 79.8 | 90.3 | — | — | — | |
| CORALGroup labels in train set?=Yes2022.09 | 79.8 | 90.3 | — | — | — | |
| DAT*Backbone=CLIP (ViT-L/14), Zero-shot=true2026.06 | 79.75 | 87.87 | 8.12 | — | — | |
| CBDebugModel Framework=PIP-Net2025.09 | 79.4 | 93.7 | — | — | — | |
| TIEBackbone=CLIP (ViT-L/14), Zero-shot=true2026.06 | 78.82 | 84.12 | 5.3 | — | — |