Counterfactual Input Evaluation on CrowS-Pairs
42.14SSSent-Debias
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Sent-DebiasBackbone=BERT2023.07 | 42.14 | — | — | — | — | — | — | — | — | — | |
| CCPABackbone=ELEATRA, prompt_template=(1, 1, 1)2023.07 | 49.06 | — | — | — | — | — | — | — | — | — | |
| CCPABackbone=DistilBERT, prompt_template=(1, 1, 1)2023.07 | 50.31 | — | — | — | — | — | — | — | — | — | |
| MABELBackbone=BERT2023.07 | 50.76 | — | — | — | — | — | — | — | — | — | |
| INLPBackbone=BERT2023.07 | 50.94 | — | — | — | — | — | — | — | — | — | |
| CCPABackbone=BERT, prompt_template=(1, 1, 1)2023.07 | 51.57 | — | — | — | — | — | — | — | — | — | |
| Auto-DebiasBackbone=BERT2023.07 | 52.83 | — | — | — | — | — | — | — | — | — | |
| ELEATRABackbone=ELEATRA2023.07 | 52.83 | — | — | — | — | — | — | — | — | — | |
| CDABackbone=BERT2023.07 | 55.35 | — | — | — | — | — | — | — | — | — | |
| DropoutBackbone=BERT2023.07 | 55.35 | — | — | — | — | — | — | — | — | — | |
| BERTBackbone=BERT2023.07 | 57.86 | — | — | — | — | — | — | — | — | — | |
| DistilBERTBackbone=DistilBERT2023.07 | 59.75 | — | — | — | — | — | — | — | — | — | |
| GalacticaParameters=120B2022.11 | 60.5 | 51.9 | 67.6 | 59.9 | 77.4 | 69 | 51.6 | 66.7 | 58.7 | 65.7 | |
| FairFilBackbone=BERT2023.07 | 62.89 | — | — | — | — | — | — | — | — | — | |
| FairSteerBackbone=Llama2-7B, Setting=Zero-shot2025.04 | 66.51 | — | — | — | — | — | — | — | — | — | |
| LLaMANumber of parameters=65B, Setting=zero-shot2023.02 | 66.6 | 70.6 | 79 | 57 | 81 | 70.1 | 64.2 | 66.7 | 77.8 | 71.5 | |
| FairSteerBackbone=Llama3-8B, Setting=Zero-shot2025.04 | 66.82 | — | — | — | — | — | — | — | — | — | |
| GPT-3Model Variant=Davinci2022.05 | 67.2 | 62.6 | 73.3 | 64.7 | 76.2 | 64.4 | 61.6 | 76.7 | 74.6 | 73.8 | |
| text-davinci-002Model Family=GPT-32022.11 | 67.2 | 62.6 | 73.3 | 64.7 | 76.2 | 64.4 | 61.6 | 76.7 | 74.6 | 73.8 | |
| GPT3Number of parameters=175B, Setting=zero-shot2023.02 | 67.2 | 62.6 | 73.3 | 64.7 | 76.2 | 64.4 | 61.6 | 76.7 | 74.6 | 73.8 | |
| Llama2-7BSetting=Zero-shot2025.04 | 67.75 | — | — | — | — | — | — | — | — | — | |
| Llama3-8BSetting=Zero-shot2025.04 | 67.83 | — | — | — | — | — | — | — | — | — | |
| FairSteerBackbone=Vicuna-7B, Setting=Zero-shot2025.04 | 68.06 | — | — | — | — | — | — | — | — | — | |
| FairSteerBackbone=Vicuna-13B, Setting=Zero-shot2025.04 | 69.3 | — | — | — | — | — | — | — | — | — | |
| FairSteerBackbone=Llama2-13B, Setting=Zero-shot2025.04 | 69.46 | — | — | — | — | — | — | — | — | — | |
| OPT-175BParameters=175B2022.05 | 69.5 | 65.7 | 68.6 | 68.6 | 78.6 | 67.8 | 62.9 | 76.7 | 76.2 | 76.2 | |
| OPTParameters=175B2022.11 | 69.5 | 65.7 | 68.6 | 68.6 | 78.6 | 67.8 | 62.9 | 76.7 | 76.2 | 76.2 | |
| OPTNumber of parameters=175B, Setting=zero-shot2023.02 | 69.5 | 65.7 | 68.6 | 68.6 | 78.6 | 67.8 | 62.9 | 76.7 | 76.2 | 76.2 | |
| Vicuna-7BSetting=Zero-shot2025.04 | 69.53 | — | — | — | — | — | — | — | — | — | |
| Vicuna-13BSetting=Zero-shot2025.04 | 69.92 | — | — | — | — | — | — | — | — | — | |
| Llama2-13BSetting=Zero-shot2025.04 | 70.93 | — | — | — | — | — | — | — | — | — | |
| FairSteerBackbone=Mistral-7B, Setting=Zero-shot2025.04 | 71.09 | — | — | — | — | — | — | — | — | — | |
| Mistral-7BSetting=Zero-shot2025.04 | 71.47 | — | — | — | — | — | — | — | — | — |