Social Bias Evaluation on FairFace
0.079MSBENDSEMbi
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| BENDSEMbiBackbone=ResNet-50, Zero-shot=false, Prompting Category=Bias prompts + input-specific prompts + labeled images2026.03 | 0.079 | 0.005 | — | |
| BENDVLMBackbone=ResNet-50, Zero-shot=false, Prompting Category=Bias prompts + input-specific prompts + labeled images2026.03 | 0.081 | 0.006 | — | |
| BENDSEMbiBackbone=ResNet-101, Zero-shot=false, Prompting Category=Bias prompts + input-specific prompts + labeled images2026.03 | 0.087 | 0.006 | — | |
| Ours (gender)Backbone=ViT-B/16, Training Dataset=FairFace, Evaluation Attribute=gender2024.11 | 0.101 | 0.032 | — | |
| BENDVLMBackbone=ResNet-101, Zero-shot=false, Prompting Category=Bias prompts + input-specific prompts + labeled images2026.03 | 0.107 | 0.01 | — | |
| ZSDEBIASBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.186 | 0.03 | — | |
| SEMbiBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.193 | 0.044 | — | |
| SEMbBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.196 | 0.041 | — | |
| SEMiBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias-agnostic + input-specific prompts2026.03 | 0.201 | 0.037 | — | |
| SEMiBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias-agnostic + input-specific prompts2026.03 | 0.206 | 0.031 | — | |
| ZSDEBIASBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.217 | 0.049 | — | |
| Original (gender)Backbone=ViT-B/16, Training Dataset=FairFace, Evaluation Attribute=gender2024.11 | 0.218 | 0.088 | — | |
| PRISMBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.245 | 0.069 | — | |
| SEMbBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.258 | 0.079 | — | |
| SEMbiBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.259 | 0.079 | — | |
| PRISMBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.282 | 0.107 | — | |
| ORTH-PROJBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.288 | 0.111 | — | |
| PRISM-MINIBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.291 | 0.113 | — | |
| BASE CLIPBackbone=ResNet-101, Zero-shot=true2026.03 | 0.335 | 0.144 | — | |
| BASE CLIPBackbone=ResNet-50, Zero-shot=true2026.03 | 0.351 | 0.17 | — | |
| Joint V-LBackbone=ViT-B/16, Zero-shot=true2026.07 | 0.353 | 0.125 | — | |
| ORTH-CALIBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.357 | 0.141 | — | |
| RG-TTABackbone=ViT-B/16, Zero-shot=true2026.07 | 0.372 | 0.167 | — | |
| ZSDEBIASBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.383 | 0.046 | — | |
| RG-TTABackbone=ViT-B/32, Zero-shot=true2026.07 | 0.389 | 0.148 | — | |
| ORTH-PROJBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.409 | 0.213 | — | |
| PRISM-MINIBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.411 | 0.215 | — | |
| BENDSEMbiBackbone=ResNet-101, Zero-shot=false, Prompting Category=Bias prompts + input-specific prompts + labeled images2026.03 | 0.425 | 0.059 | — | |
| ROBOSHOTBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias-agnostic + input-specific prompts2026.03 | 0.445 | 0.299 | — | |
| BENDSEMbiBackbone=ResNet-50, Zero-shot=false, Prompting Category=Bias prompts + input-specific prompts + labeled images2026.03 | 0.455 | 0.067 | — | |
| ORTH-CALIBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.47 | 0.278 | — | |
| Ours (race)Backbone=ViT-B/16, Training Dataset=FairFace, Evaluation Attribute=race2024.11 | 0.471 | 0.142 | — | |
| ROBOSHOTBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias-agnostic + input-specific prompts2026.03 | 0.494 | 0.338 | — | |
| Joint V-LBackbone=ViT-B/32, Zero-shot=true2026.07 | 0.503 | 0.149 | — | |
| Biased-promptsBackbone=ViT-B/16, Zero-shot=true2026.07 | 0.518 | 0.219 | — | |
| Original (race)Backbone=ViT-B/16, Training Dataset=FairFace, Evaluation Attribute=race2024.11 | 0.528 | 0.182 | — | |
| Original CLIPBackbone=ViT-B/16, Zero-shot=true2026.07 | 0.528 | 0.182 | — | |
| CLIP-clipBackbone=ViT-B/16, Zero-shot=true2026.07 | 0.544 | 0.161 | — | |
| SEMiBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias-agnostic + input-specific prompts2026.03 | 0.563 | 0.126 | — | |
| SEMiBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias-agnostic + input-specific prompts2026.03 | 0.566 | 0.111 | — | |
| Original CLIPBackbone=ViT-B/32, Zero-shot=true2026.07 | 0.568 | 0.165 | — | |
| BENDVLMBackbone=ResNet-50, Zero-shot=false, Prompting Category=Bias prompts + input-specific prompts + labeled images2026.03 | 0.581 | 0.15 | — | |
| BENDVLMBackbone=ResNet-101, Zero-shot=false, Prompting Category=Bias prompts + input-specific prompts + labeled images2026.03 | 0.583 | 0.125 | — | |
| ZSDEBIASBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.588 | 0.082 | — | |
| PRISMBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.594 | 0.152 | — | |
| Biased-promptsBackbone=ViT-B/32, Zero-shot=true2026.07 | 0.595 | 0.282 | — | |
| SEMbiBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.623 | 0.14 | — | |
| SEMbiBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.626 | 0.153 | — | |
| PRISMBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.632 | 0.157 | — | |
| SEMbBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.638 | 0.152 | — | |
| Ours (age)Backbone=ViT-B/16, Training Dataset=FairFace, Evaluation Attribute=age2024.11 | 0.649 | 0.425 | — | |
| SEMbBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.652 | 0.171 | — | |
| Original (age)Backbone=ViT-B/16, Training Dataset=FairFace, Evaluation Attribute=age2024.11 | 0.657 | 0.433 | — | |
| ROBOSHOTBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias-agnostic + input-specific prompts2026.03 | 0.706 | 0.215 | — | |
| CLIP-clipBackbone=ViT-B/32, Zero-shot=true2026.07 | 0.713 | 0.227 | — | |
| BASE CLIPBackbone=ResNet-50, Zero-shot=true2026.03 | 0.735 | 0.215 | — | |
| BASE CLIPBackbone=ResNet-101, Zero-shot=true2026.03 | 0.744 | 0.203 | — | |
| ROBOSHOTBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias-agnostic + input-specific prompts2026.03 | 0.798 | 0.222 | — | |
| PRISM-MINIBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.837 | 0.313 | — | |
| ORTH-CALIBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.842 | 0.297 | — | |
| ORTH-PROJBackbone=ResNet-101, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.843 | 0.322 | — | |
| ORTH-CALIBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts + input-specific prompts2026.03 | 0.91 | 0.411 | — | |
| PRISM-MINIBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.983 | 0.454 | — | |
| ORTH-PROJBackbone=ResNet-50, Zero-shot=true, Prompting Category=Bias prompts only2026.03 | 0.996 | 0.464 | — |