Object Classification on Waterbirds (test)
90Worst-Group AccuracyUMIX
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UMIXGroup-oblivious setting=true2022.09 | 90 | 93 | — | |
| SSAModel=ResNet-50, # Trained Params=25557032, % Params=1002022.07 | 89 | — | 0 | |
| CNCModel=ResNet-50, # Trained Params=25557032, % Params=1002022.07 | 88.5 | — | -0.5 | |
| Contrastive Adapter (CLIP ViT-L/14)Model=Adapter + CLIP ViT-L/14, # Trained Params=197632, % Params=0.772022.07 | 86.9 | — | -2.1 | |
| JTTModel=ResNet-50, # Trained Params=25557032, % Params=1002022.07 | 86.7 | — | -2.3 | |
| RWYModel=ResNet-50, # Trained Params=25557032, % Params=1002022.07 | 86.1 | — | -2.9 | |
| JTTGroup-oblivious setting=true2022.09 | 86 | 93.6 | — | |
| Contrastive Adapter (CLIP RN-50)Model=Adapter + CLIP RN-50, # Trained Params=263424, % Params=1.032022.07 | 83.7 | — | -5.3 | |
| CIMModel=ResNet-50, # Trained Params=25557032, % Params=1002022.07 | 83.6 | — | -5.4 | |
| SUBYModel=ResNet-50, # Trained Params=25557032, % Params=1002022.07 | 82.4 | — | -6.6 | |
| Group DROBias Supervision=Supervised2021.08 | 80.82 | 88.99 | — | |
| EIILModel=ResNet-50, # Trained Params=25557032, % Params=1002022.07 | 78.7 | — | -10.3 | |
| CVaR-DROGroup-oblivious setting=true2022.09 | 77.2 | 90.3 | — | |
| CVaR-DOROGroup-oblivious setting=true2022.09 | 77 | 91.5 | — | |
| chi^2-DOROGroup-oblivious setting=true2022.09 | 76 | 89.5 | — | |
| chi^2-DROGroup-oblivious setting=true2022.09 | 74 | 88.8 | — | |
| Focal LossGroup-oblivious setting=true2022.09 | 73.1 | 87 | — | |
| BPABias Supervision=Unsupervised2021.08 | 71.39 | 87.05 | — | |
| LfFBias Supervision=Unsupervised2021.08 | 68.02 | 85.48 | — | |
| ERMGroup-oblivious setting=true2022.09 | 63.7 | 97 | — | |
| BaseBias Supervision=Unsupervised2021.08 | 62.39 | 84.63 | — | |
| LfF*Bias Supervision=Unsupervised, Fine-tuned=Classification layer only2021.08 | 61.68 | 84.57 | — |