Image Classification on ColoredMNIST (test)
0.9395Average AccuracyBiasEdit
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| BiasEditCheckpoint selection strategy=best in-distribution validation performance, Training bias-conflict ratio=0%2026.05 | 0.9395 | — | — | — | 91.28 | 96.62 | — | |
| MBDG2021.02 | 0.685 | 0.737 | 0.684 | 0.635 | — | — | — | |
| MBDG-DA2021.02 | 0.59 | 0.727 | 0.714 | 0.332 | — | — | — | |
| MBDG-Reg2021.02 | 0.581 | 0.733 | 0.737 | 0.272 | — | — | — | |
| BiasAdvCheckpoint selection strategy=best in-distribution validation performance, Training bias-conflict ratio=0%2026.05 | 0.5143 | — | — | — | 2.88 | 99.97 | — | |
| SelecMixCheckpoint selection strategy=best in-distribution validation performance, Training bias-conflict ratio=0%2026.05 | 0.5108 | — | — | — | 2.22 | 99.93 | — | |
| LfFCheckpoint selection strategy=best in-distribution validation performance, Training bias-conflict ratio=0%2026.05 | 0.5055 | — | — | — | 1.13 | 99.98 | — | |
| DFACheckpoint selection strategy=best in-distribution validation performance, Training bias-conflict ratio=0%2026.05 | 0.504 | — | — | — | 0.81 | 100 | — | |
| LfF+BECheckpoint selection strategy=best in-distribution validation performance, Training bias-conflict ratio=0%2026.05 | 0.5033 | — | — | — | 0.66 | 100 | — | |
| VanillaCheckpoint selection strategy=best in-distribution validation performance, Training bias-conflict ratio=0%2026.05 | 0.5026 | — | — | — | 0.56 | 99.96 | — | |
| SoftConCheckpoint selection strategy=best in-distribution validation performance, Training bias-conflict ratio=0%2026.05 | 0.5006 | — | — | — | 0.12 | 100 | — | |
| DeNetDMCheckpoint selection strategy=best in-distribution validation performance, Training bias-conflict ratio=0%2026.05 | 0.5 | — | — | — | 0.01 | 100 | — | |
| MBDA2021.02 | 0.483 | 0.72 | 0.507 | 0.225 | — | — | — | |
| DANN2021.02 | 0.407 | 0.499 | 0.621 | 0.1 | — | — | — | |
| IRM2021.02 | 0.403 | 0.467 | 0.512 | 0.231 | — | — | — | |
| CORAL2021.02 | 0.397 | 0.495 | 0.595 | 0.102 | — | — | — | |
| CDANN2021.02 | 0.391 | 0.632 | 0.444 | 0.099 | — | — | — | |
| VREX2021.02 | 0.369 | 0.502 | 0.505 | 0.101 | — | — | — | |
| GroupDRO2021.02 | 0.368 | 0.501 | 0.5 | 0.102 | — | — | — | |
| MMD2021.02 | 0.368 | 0.503 | 0.5 | 0.099 | — | — | — | |
| ARM2021.02 | 0.368 | 0.5 | 0.501 | 0.102 | — | — | — | |
| ERM2021.02 | 0.367 | 0.5 | 0.501 | 0.1 | — | — | — | |
| MLDG2021.02 | 0.367 | 0.501 | 0.501 | 0.1 | — | — | — | |
| SagNet2021.02 | 0.365 | 0.499 | 0.497 | 0.1 | — | — | — | |
| RSC2021.02 | 0.365 | 0.496 | 0.497 | 0.101 | — | — | — | |
| MTL2021.02 | 0.35 | 0.443 | 0.507 | 0.101 | — | — | — | |
| Mixup2021.02 | 0.334 | 0.366 | 0.534 | 0.102 | — | — | — | |
| DIAR-RA2025.02 | — | — | — | — | — | — | 73.1 | |
| ERM2025.02 | — | — | — | — | — | — | 16.1 | |
| ERM grayscaletype=oracle2025.02 | — | — | — | — | — | — | 73.1 | |
| Fishr2025.02 | — | — | — | — | — | — | 68.8 | |
| IPG2025.02 | — | — | — | — | — | — | 72.8 | |
| IRM2025.02 | — | — | — | — | — | — | 66.9 | |
| meta-IRM2025.02 | — | — | — | — | — | — | 70.4 | |
| NCM2025.02 | — | — | — | — | — | — | 69.3 | |
| Optimal2025.02 | — | — | — | — | — | — | 75 | |
| Random2025.02 | — | — | — | — | — | — | 50 | |
| Shock Graph2025.02 | — | — | — | — | — | — | 71.6 |