Table Question Answering on ROBUT-SQA Table Content Perturbations
68.3Pre-perturbation AccuracyTAPEX
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TAPEXPerturbation Type=Column Masking2023.06 | 68.3 | 65 | 3.3 | 87.4 | |
| TAPEXPerturbation Type=Column Adding2023.06 | 67.3 | 58.9 | 8.4 | 81.9 | |
| TAPEXPerturbation Type=Column Extension2023.06 | 66.8 | 42 | 24.8 | 58.9 | |
| TAPEXPerturbation Type=Column Order Shuffling2023.06 | 66.7 | 65.2 | 1.5 | 95.8 | |
| TAPASPerturbation Type=Column Masking2023.06 | 65.2 | 62.3 | 2.9 | 89.6 | |
| TAPEXPerturbation Type=Row Order Shuffling2023.06 | 64.1 | 60.2 | 3.9 | 88.9 | |
| TAPASPerturbation Type=Row Order Shuffling2023.06 | 63.3 | 55.3 | 8 | 85 | |
| TAPASPerturbation Type=Column Adding2023.06 | 62.7 | 60.8 | 1.9 | 93.7 | |
| TAPASPerturbation Type=Column Extension2023.06 | 62.4 | 40.8 | 21.6 | 62.1 | |
| TAPASPerturbation Type=Column Order Shuffling2023.06 | 61.3 | 60.5 | 0.8 | 94.8 | |
| GPT-3few-shot=true, Perturbation Type=Column Masking2023.06 | 51.3 | 49.3 | 2 | 89.6 | |
| GPT-3few-shot=true, Perturbation Type=Column Adding2023.06 | 50.2 | 48.5 | 1.7 | 91.6 | |
| GPT-3few-shot=true, Perturbation Type=Column Order Shuffling2023.06 | 49.5 | 49 | 0.5 | 94.5 | |
| GPT-3few-shot=true, Perturbation Type=Column Extension2023.06 | 49.5 | 34.9 | 14.6 | 60.8 | |
| GPT-3few-shot=true, Perturbation Type=Row Order Shuffling2023.06 | 49.2 | 47.4 | 1.8 | 93.7 |