# Multiple-Debias: A Full-process Debiasing Method for Multilingual Pre-trained Language Models

Source: [Multiple-Debias: A Full-process Debiasing Method for Multilingual Pre-trained Language Models](<https://www.sota2.com/research/paper/69d33dd97d0172140ec734ee>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-07T05:00:04.607Z

Authors: Haoyu Liang, Peijian Zeng, Wentao Huang, Aimin Yang, Dong Zhou

Published: 2026-04-03T06:30:06.000Z

## Abstract

Multilingual Pre-trained Language Models (MPLMs) have become essential tools for natural language processing. However, they often exhibit biases related to sensitive attributes such as gender, race, and religion. In this paper, we introduce a comprehensive multilingual debiasing method named Multiple-Debias to address these issues across multiple languages. By incorporating multilingual counterfactual data augmentation and multilingual Self-Debias across both pre-processing and post-processing stages, alongside parameter-efficient fine-tuning, we significantly reduced biases in MPLMs across three sensitive attributes in four languages. We also extended CrowS-Pairs to German, Spanish, Chinese, and Japanese, validating our full-process multilingual debiasing method for gender, racial, and religious bias. Our experiments show that (i) multilingual debiasing methods surpass monolingual approaches in effectively mitigating biases, and (ii) integrating debiasing information from different languages notably improves the fairness of MPLMs.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Gender Bias Mitigation on Multilingual CrowS-Pairs](<https://www.sota2.com/research/sota/gender-bias-mitigation-on-multilingual-crows-pairs.md>): Gender Bias Mitigation; Multilingual CrowS-Pairs gender-sensitive attributes; Bias Score (DE): 0.83.

- [Religious Bias Evaluation on Multilingual CrowS-Pairs (test)](<https://www.sota2.com/research/sota/religious-bias-evaluation-on-multilingual-crows-pairs-test.md>): Religious Bias Evaluation; Multilingual CrowS-Pairs (test); Bias Score (DE): 4.17.

- [Racial Bias Evaluation on Multilingual CrowS-Pairs](<https://www.sota2.com/research/sota/racial-bias-evaluation-on-multilingual-crows-pairs.md>): Racial Bias Evaluation; Multilingual CrowS-Pairs racial bias; Bias Score (DE): 11.41.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.02772>)

[PDF](<https://arxiv.org/pdf/2604.02772.pdf>)