# Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods

Source: [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](<https://www.sota2.com/research/paper/68fa643f1e209131cea03715>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:59:11.405Z

Authors: Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang

Published: 2018-04-18T18:51:00.000Z

## Abstract

We introduce a new benchmark, WinoBias, for coreference resolution focused on gender bias. Our corpus contains Winograd-schema style sentences with entities corresponding to people referred by their occupation (e.g. the nurse, the doctor, the carpenter). We demonstrate that a rule-based, a feature-rich, and a neural coreference system all link gendered pronouns to pro-stereotypical entities with higher accuracy than anti-stereotypical entities, by an average difference of 21.1 in F1 score. Finally, we demonstrate a data-augmentation approach that, in combination with existing word-embedding debiasing techniques, removes the bias demonstrated by these systems in WinoBias without significantly affecting their performance on existing coreference benchmark datasets. Our dataset and code are available at http://winobias.org.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Gender Bias Mitigation on Multilingual CrowS-Pairs](<https://www.sota2.com/research/sota/gender-bias-mitigation-on-multilingual-crows-pairs.md>): Gender Bias Mitigation; Multilingual CrowS-Pairs gender-sensitive attributes; Bias Score (DE): 1.37.

- [Racial Bias Evaluation on Multilingual CrowS-Pairs](<https://www.sota2.com/research/sota/racial-bias-evaluation-on-multilingual-crows-pairs.md>): Racial Bias Evaluation; Multilingual CrowS-Pairs racial bias; Bias Score (DE): 15.56.

- [Religious Bias Evaluation on Multilingual CrowS-Pairs (test)](<https://www.sota2.com/research/sota/religious-bias-evaluation-on-multilingual-crows-pairs-test.md>): Religious Bias Evaluation; Multilingual CrowS-Pairs (test); Bias Score (DE): 16.67.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1804.06876>)

[PDF](<https://arxiv.org/pdf/1804.06876.pdf>)