# Toward Deconfounding the Influence of Entity Demographics for Question Answering Accuracy

Source: [Toward Deconfounding the Influence of Entity Demographics for Question Answering Accuracy](<https://www.sota2.com/research/paper/68fa6ab31e209131cea29d93>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:31:18.149Z

Authors: Maharshi Gor, Kellie Webster, Jordan Boyd-Graber

Published: 2021-04-15T16:26:54.000Z

## Abstract

The goal of question answering (QA) is to answer any question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, model accuracy analysis reveals little evidence that accuracy is lower for people based on gender or nationality; instead, there is more variation on professions (question topic). But QA's lack of representation could itself hide evidence of bias, necessitating QA datasets that better represent global diversity.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2104.07571>)

[PDF](<https://arxiv.org/pdf/2104.07571.pdf>)