# RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations

Source: [RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations](<https://www.sota2.com/research/paper/68fa6fdc1e209131cea47b48>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:34:24.440Z

Authors: Yilun Zhao, Chen Zhao, Linyong Nan, Zhenting Qi, Wenlin Zhang, Xiangru Tang, Boyu Mi, Dragomir Radev

Published: 2023-06-25T19:23:21.000Z

## Abstract

Despite significant progress having been made in question answering on tabular data (Table QA), it's unclear whether, and to what extent existing Table QA models are robust to task-specific perturbations, e.g., replacing key question entities or shuffling table columns. To systematically study the robustness of Table QA models, we propose a benchmark called RobuT, which builds upon existing Table QA datasets (WTQ, WikiSQL-Weak, and SQA) and includes human-annotated adversarial perturbations in terms of table header, table content, and question. Our results indicate that both state-of-the-art Table QA models and large language models (e.g., GPT-3) with few-shot learning falter in these adversarial sets. We propose to address this problem by using large language models to generate adversarial examples to enhance training, which significantly improves the robustness of Table QA models. Our data and code is publicly available at https://github.com/yilunzhao/RobuT.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Table Question Answering on ROBUT-SQA Table Content Perturbations](<https://www.sota2.com/research/sota/table-question-answering-on-robut-sqa-table-content-perturbations.md>): Table Question Answering; ROBUT-SQA Table Content Perturbations; Pre-perturbation Accuracy: not recorded.

- [Table Question Answering on ROBUT-WTQ Synonym Replacement (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wtq-synonym-replacement-test.md>): Table Question Answering; ROBUT-WTQ Synonym Replacement (test); Accuracy (Pre-perturbation): not recorded.

- [Table Question Answering on ROBUT-WTQ Abbreviation Replacement (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wtq-abbreviation-replacement-test.md>): Table Question Answering; ROBUT-WTQ Abbreviation Replacement (test); Accuracy (Pre-perturbation): not recorded.

- [Table Question Answering on ROBUT-WTQ Row Order Shuffling (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wtq-row-order-shuffling-test.md>): Table Question Answering; ROBUT-WTQ Row Order Shuffling (test); Accuracy (Pre-perturbation): not recorded.

- [Table Question Answering on ROBUT-WTQ Column Order Shuffling (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wtq-column-order-shuffling-test.md>): Table Question Answering; ROBUT-WTQ Column Order Shuffling (test); Accuracy (Pre-perturbation): not recorded.

- [Table Question Answering on ROBUT-WTQ Column Extension (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wtq-column-extension-test.md>): Table Question Answering; ROBUT-WTQ Column Extension (test); Accuracy (Pre-perturbation): not recorded.

- [Table Question Answering on ROBUT-WTQ Column Adding (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wtq-column-adding-test.md>): Table Question Answering; ROBUT-WTQ Column Adding (test); Accuracy (Pre-Perturbation): not recorded.

- [Table Question Answering on ROBUT-WTQ Word-Level Paraphrase (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wtq-word-level-paraphrase-test.md>): Table Question Answering; ROBUT-WTQ Word-Level Paraphrase (test); Accuracy (Pre-Perturbation): not recorded.

- [Table Question Answering on ROBUT-WTQ Sentence-Level Paraphrase (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wtq-sentence-level-paraphrase-test.md>): Table Question Answering; ROBUT-WTQ Sentence-Level Paraphrase (test); Accuracy (Pre-perturbation): not recorded.

- [Table Question Answering on ROBUT-SQA Table Header Perturbations](<https://www.sota2.com/research/sota/table-question-answering-on-robut-sqa-table-header-perturbations.md>): Table Question Answering; ROBUT-SQA Table Header Perturbations; Pre-perturbation Accuracy: not recorded.

- [Table Question Answering on ROBUT-SQA Natural Language Query Perturbations](<https://www.sota2.com/research/sota/table-question-answering-on-robut-sqa-natural-language-query-perturbations.md>): Table Question Answering; ROBUT-SQA Natural Language Query Perturbations; Pre-perturbation Accuracy: not recorded.

- [Table Question Answering on ROBUT-WTQ (dev)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wtq-dev.md>): Table Question Answering; ROBUT-WTQ (dev); Accuracy: not recorded.

- [Table Question Answering on WTQ Column Masking ROBUT (test)](<https://www.sota2.com/research/sota/table-question-answering-on-wtq-column-masking-robut-test.md>): Table Question Answering; WTQ Column Masking ROBUT (test); Accuracy (Pre-Perturbation): not recorded.

- [Table Question Answering on WTQ Mix ROBUT (test)](<https://www.sota2.com/research/sota/table-question-answering-on-wtq-mix-robut-test.md>): Table Question Answering; WTQ Mix ROBUT (test); Accuracy (Pre-perturbation): not recorded.

- [Table Question Answering on WIKISQL ROBUT (dev)](<https://www.sota2.com/research/sota/table-question-answering-on-wikisql-robut-dev.md>): Table Question Answering; WIKISQL ROBUT (dev); Accuracy: not recorded.

- [Table Question Answering on ROBUT-WIKISQL Column Masking (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wikisql-column-masking-test.md>): Table Question Answering; ROBUT-WIKISQL Column Masking (test); Accuracy (Pre-perturbation): not recorded.

- [Table Question Answering on ROBUT-WIKISQL Mix (test)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-wikisql-mix-test.md>): Table Question Answering; ROBUT-WIKISQL Mix (test); Accuracy (Pre-perturbation): not recorded.

- [Table Question Answering on ROBUT-SQA (dev)](<https://www.sota2.com/research/sota/table-question-answering-on-robut-sqa-dev.md>): Table Question Answering; ROBUT-SQA (dev); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2306.14321>)

[PDF](<https://arxiv.org/pdf/2306.14321.pdf>)

[Implementation](<https://github.com/yilunzhao/RobuT>)
