# Comparative Analysis of Neural QA models on SQuAD

Source: [Comparative Analysis of Neural QA models on SQuAD](<https://www.sota2.com/research/paper/68fa647b1e209131cea04bc5>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:04:03.574Z

Authors: Soumya Wadhwa, Khyathi Raghavi Chandu, Eric Nyberg

Published: 2018-06-18T22:29:51.000Z

## Abstract

The task of Question Answering has gained prominence in the past few decades for testing the ability of machines to understand natural language. Large datasets for Machine Reading have led to the development of neural models that cater to deeper language understanding compared to information retrieval tasks. Different components in these neural architectures are intended to tackle different challenges. As a first step towards achieving generalization across multiple domains, we attempt to understand and compare the peculiarities of existing end-to-end neural models on the Stanford Question Answering Dataset (SQuAD) by performing quantitative as well as qualitative analysis of the results attained by each of them. We observed that prediction errors reflect certain model-specific biases, which we further discuss in this paper.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1806.06972>)

[PDF](<https://arxiv.org/pdf/1806.06972.pdf>)