# Investigating Biases in Textual Entailment Datasets

Source: [Investigating Biases in Textual Entailment Datasets](<https://www.sota2.com/research/paper/68fa65f81e209131cea0d91d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:40:30.056Z

Authors: Shawn Tan, Yikang Shen, Chin-wei Huang, Aaron Courville

Published: 2019-06-23T19:38:53.000Z

## Abstract

The ability to understand logical relationships between sentences is an important task in language understanding. To aid in progress for this task, researchers have collected datasets for machine learning and evaluation of current systems. However, like in the crowdsourced Visual Question Answering (VQA) task, some biases in the data inevitably occur. In our experiments, we find that performing classification on just the hypotheses on the SNLI dataset yields an accuracy of 64%. We analyze the bias extent in the SNLI and the MultiNLI dataset, discuss its implication, and propose a simple method to reduce the biases in the datasets.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1906.09635>)

[PDF](<https://arxiv.org/pdf/1906.09635.pdf>)
