Gender discrimination risk evaluation on Occupational Contexts (test)
0R ScoreIdeally Unbiased
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Ideally UnbiasedDescription=Baseline model consistently delivering unbiased predictions2024.02 | 0 | 0 | 0 | |
| BERTparameters=110M2024.02 | 0.3049 | 0.3018 | 0.0031 | |
| ALBERTparameters=11.8M2024.02 | 0.3287 | 0.2531 | 0.0756 | |
| GPT-2parameters=137M2024.02 | 0.4157 | 0.3956 | 0.02 | |
| ROBERTaparameters=125M2024.02 | 0.4535 | 0.4171 | 0.0364 | |
| ChatGLM2parameters=6B2024.02 | 0.4792 | 0.4504 | 0.0288 | |
| Baichuan2024.02 | 0.4831 | 0.4703 | 0.0134 | |
| BARTparameters=139M2024.02 | 0.4846 | 0.4677 | 0.0169 | |
| OPT-IML2024.02 | 0.5049 | 0.487 | 0.0178 | |
| distilBERTparameters=67M2024.02 | 0.5078 | 0.4914 | 0.0164 | |
| LLAMA22024.02 | 0.708 | 0.7 | 0.008 | |
| XLNetparameters=117M2024.02 | 0.7343 | 0.7177 | 0.0166 | |
| T5parameters=220M2024.02 | 0.8703 | 0.8691 | 0.0012 | |
| StereotypedDescription=Baseline model adhering to extreme unipolar stereotypes2024.02 | 1 | 1 | 0 | |
| Randomly StereotypedDescription=Baseline model with random behavior across contexts2024.02 | 1 | 0 | 1 |