# Semantic Parsing on WIKITABLEQUESTIONS (test) Source: [Semantic Parsing on WIKITABLEQUESTIONS (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:42:33.849Z Evaluation of Semantic Parsing performance on the test split of the WIKITABLEQUESTIONS dataset, typically measured using accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Semantic Parsing | | Dataset (including any stated split) | WIKITABLEQUESTIONS (test) | | Metric | Execution Accuracy (Best) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Execution Accuracy (Best)](): Higher is better; unit: PERCENT. - [Accuracy](): Higher is better; unit: PERCENT. - [Execution Accuracy (Mean)](): Higher is better; unit: PERCENT. - [Execution Accuracy (Std Dev)](): Lower is better; unit: SCALAR. ## Recorded results | Method | Execution Accuracy (Best) | Result date | Report paper | | --- | --- | --- | --- | | TaBERT-Large | 52.3 | Not recorded | 68fa678b1e209131cea16d41 | | TaBERT-Large | 51.5 | Not recorded | 68fa678b1e209131cea16d41 | | TaBERT-Base | 51.3 | Not recorded | 68fa678b1e209131cea16d41 | | TaBERT-Base | 51.2 | Not recorded | 68fa678b1e209131cea16d41 | | MAPO w/ BERT-Base | 50.2 | Not recorded | 68fa678b1e209131cea16d41 | | MAPO w/ BERT-Large | 50.1 | Not recorded | 68fa678b1e209131cea16d41 | | TaBERT-Base | 50 | Not recorded | 68fa678b1e209131cea16d41 | | MAPO w/ BERT-Base | 49.2 | Not recorded | 68fa678b1e209131cea16d41 | | Ensemble 10 Models | 46.9 | Not recorded | 68fa678b1e209131cea16d41 | | Wang et al. (2019b) | 44.5 | Not recorded | 68fa678b1e209131cea16d41 | | Dasigi et al. (2019) | 44.3 | Not recorded | 68fa678b1e209131cea16d41 | | Agarwal et al. (2019) | 44.1 | Not recorded | 68fa678b1e209131cea16d41 | | Base Parser (MAPO) | 43.8 | Not recorded | 68fa678b1e209131cea16d41 | | Zhang et al. (2017) | 43.7 | Not recorded | 68fa678b1e209131cea16d41 | | Ensemble 15 Models | 37.7 | Not recorded | 68fa678b1e209131cea16d41 | | Pasupat and Liang (2015) | 37.1 | Not recorded | 68fa678b1e209131cea16d41 | | Neelakantan et al. (2016) | 34.2 | Not recorded | 68fa678b1e209131cea16d41 | ### TaBERT-Large Execution Accuracy (Best): 52.3. Result ID: 698d39d38b517eb4331fea95. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"TaBERT-Large","K (number of rows)":3,"content\_snapshot":"included","base\_system":"MAPO"}. This record does not assert independent reproduction. ### TaBERT-Large Execution Accuracy (Best): 51.5. Result ID: 698d39d38b517eb4331fea94. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"TaBERT-Large","K (number of rows)":1,"content\_snapshot":"included","base\_system":"MAPO"}. This record does not assert independent reproduction. ### TaBERT-Base Execution Accuracy (Best): 51.3. Result ID: 698d39d38b517eb4331fea92. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"TaBERT-Base","K (number of rows)":3,"content\_snapshot":"included","base\_system":"MAPO"}. This record does not assert independent reproduction. ### TaBERT-Base Execution Accuracy (Best): 51.2. Result ID: 698d39d38b517eb4331fea90. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"TaBERT-Base","K (number of rows)":1,"content\_snapshot":"included","base\_system":"MAPO"}. This record does not assert independent reproduction. ### MAPO w/ BERT-Base Execution Accuracy (Best): 50.2. Result ID: 698d39d38b517eb4331fea8f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"BERT-Base","K (number of rows)":1,"content\_snapshot":"excluded"}. This record does not assert independent reproduction. ### MAPO w/ BERT-Large Execution Accuracy (Best): 50.1. Result ID: 698d39d38b517eb4331fea93. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"BERT-Large","K (number of rows)":1,"content\_snapshot":"included"}. This record does not assert independent reproduction. ### TaBERT-Base Execution Accuracy (Best): 50. Result ID: 698d39d38b517eb4331fea91. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"backbone":"TaBERT-Base","K (number of rows)":1,"content\_snapshot":"excluded","base\_system":"MAPO"}. This record does not assert independent reproduction. ### MAPO w/ BERT-Base Execution Accuracy (Best): 49.2. Result ID: 698d39d38b517eb4331fea8e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"backbone":"BERT-Base","K (number of rows)":1,"content\_snapshot":"included"}. This record does not assert independent reproduction. ### Ensemble 10 Models Execution Accuracy (Best): 46.9. Result ID: 698d39d38b517eb4331fea8b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Ensemble size":"10"}. This record does not assert independent reproduction. ### Wang et al. (2019b) Execution Accuracy (Best): 44.5. Result ID: 698d39d38b517eb4331fea8c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Dasigi et al. (2019) Execution Accuracy (Best): 44.3. Result ID: 698d39d38b517eb4331fea89. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Agarwal et al. (2019) Execution Accuracy (Best): 44.1. Result ID: 698d39d38b517eb4331fea8a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Base Parser (MAPO) Execution Accuracy (Best): 43.8. Result ID: 698d39d38b517eb4331fea8d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"base\_system":"MAPO"}. This record does not assert independent reproduction. ### Zhang et al. (2017) Execution Accuracy (Best): 43.7. Result ID: 698d39d38b517eb4331fea88. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Ensemble 15 Models Execution Accuracy (Best): 37.7. Result ID: 698d39d38b517eb4331fea87. [Reporting paper]() Recorded evaluation configuration: {"Ensemble size":"15"}. This record does not assert independent reproduction. ### Pasupat and Liang (2015) Execution Accuracy (Best): 37.1. Result ID: 698d39d38b517eb4331fea85. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Neelakantan et al. (2016) Execution Accuracy (Best): 34.2. Result ID: 698d39d38b517eb4331fea86. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.