Stereotypical Bias Evaluation on StereoSet (dev)
84.172Overall LMS ScoreNone
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NoneType=Baselines, training samples=1002023.06 | 84.172 | — | — | — | — | — | — | — | — | — | — | — | — | 60.279 | 70.3 | |
| SentenceDebiasType=Baselines, training samples=1002023.06 | 84.166 | — | — | — | — | — | — | — | — | — | — | — | — | 59.221 | 71.308 | |
| neutral-maskingType=Ours, training samples=102023.06 | 83.956 | — | — | — | — | — | — | — | — | — | — | — | — | 60.341 | 72.548 | |
| DropoutType=Baselines, training samples=1002023.06 | 83.811 | — | — | — | — | — | — | — | — | — | — | — | — | 60.529 | 70.171 | |
| neutral-maskingType=Ours, training samples=1002023.06 | 83.587 | — | — | — | — | — | — | — | — | — | — | — | — | 60.814 | 72.213 | |
| CDAType=Baselines, training samples=1002023.06 | 83.466 | — | — | — | — | — | — | — | — | — | — | — | — | 60.022 | 70.892 | |
| INLPType=Baselines, training samples=1002023.06 | 83.391 | — | — | — | — | — | — | — | — | — | — | — | — | 58.205 | 70.966 | |
| random-phrase-maskingType=Ours, training samples=102023.06 | 80.312 | — | — | — | — | — | — | — | — | — | — | — | — | 59.442 | 70.406 | |
| random-phrase-maskingType=Ours, training samples=1002023.06 | 78.676 | — | — | — | — | — | — | — | — | — | — | — | — | 58.037 | 69.949 | |
| text-davinci-0022022.11 | 77.6 | 78.4 | 63.4 | 57.5 | 75.6 | 66.5 | 50.6 | 80.8 | 59 | 66.3 | 77 | 57.4 | 65.7 | 60.8 | 60.8 | |
| GalacticaParameters=120B2022.11 | 75 | 75.2 | 57.2 | 64.3 | 74.6 | 59.1 | 61 | 81.4 | 55.1 | 73.1 | 74.5 | 54.8 | 67.3 | 56.2 | 65.6 | |
| OPTParameters=175B2022.11 | 74.8 | 74.1 | 62.6 | 55.4 | 74 | 63.6 | 53.8 | 84 | 59 | 68.9 | 74.9 | 56.8 | 64.8 | 59.9 | 60 |