Semantic Parsing on WIKITABLEQUESTIONS (dev)
53Execution Accuracy (Best)TaBERT-Large
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TaBERT-Largebackbone=TaBERT-Large, K (number of rows)=3, content_snapshot=included, base_system=MAPO2020.05 | 53 | — | 52.2 | 0.7 | |
| TaBERT-Largebackbone=TaBERT-Large, K (number of rows)=1, content_snapshot=included, base_system=MAPO2020.05 | 52.7 | — | 51.6 | 1.1 | |
| TaBERT-Basebackbone=TaBERT-Base, K (number of rows)=3, content_snapshot=included, base_system=MAPO2020.05 | 52.4 | — | 51.6 | 0.5 | |
| TaBERT-Basebackbone=TaBERT-Base, K (number of rows)=1, content_snapshot=included, base_system=MAPO2020.05 | 51.6 | — | 51.2 | 0.5 | |
| MAPO w/ BERT-Largebackbone=BERT-Large, K (number of rows)=1, content_snapshot=included2020.05 | 50.8 | — | 50.3 | 0.4 | |
| MAPO w/ BERT-Basebackbone=BERT-Base, K (number of rows)=1, content_snapshot=included2020.05 | 50.4 | — | 49.6 | 0.5 | |
| TaBERT-Basebackbone=TaBERT-Base, K (number of rows)=1, content_snapshot=excluded, base_system=MAPO2020.05 | 50.3 | — | 49.9 | 0.4 | |
| MAPO w/ BERT-Basebackbone=BERT-Base, K (number of rows)=1, content_snapshot=excluded2020.05 | 50 | — | 49.1 | 0.6 | |
| Wang et al. (2019b)2020.05 | 43.7 | — | — | — | |
| Agarwal et al. (2019)2020.05 | 43.2 | — | — | — | |
| Dasigi et al. (2019)2020.05 | 43.1 | — | — | — | |
| Base Parser (MAPO)base_system=MAPO2020.05 | 42.7 | — | 42.3 | 0.3 | |
| Zhang et al. (2017)2020.05 | 40.6 | — | — | — | |
| Ensemble 15 ModelsEnsemble size=152020.05 | 37.5 | — | — | — | |
| Pasupat and Liang (2015)2020.05 | 37 | — | — | — | |
| Neelakantan et al. (2016)2020.05 | 34.1 | — | — | — | |
| Abstract Programs f.w. standard attentionSupervision=Denotations, Attention=standard2019.09 | — | 39.4 | — | — | |
| Agarwal et al. (2019)Supervision=Denotations2019.09 | — | 43.2 | — | — | |
| Dasigi et al. (2019)Supervision=Denotations2019.09 | — | 42.1 | — | — | |
| Liang et al. (2018)Supervision=Denotations2019.09 | — | 42.3 | — | — | |
| Neelakantan et al. (2017)Supervision=Denotations2019.09 | — | 34.1 | — | — | |
| Pasupat and Liang (2015)Supervision=Denotations2019.09 | — | 37 | — | — | |
| Structured-Attention ParserSupervision=Denotations, Attention=structured2019.09 | — | 43.7 | — | — | |
| Typed Seq2SeqSupervision=Denotations2019.09 | — | 37.3 | — | — | |
| Zhang et al. (2017)Supervision=Denotations2019.09 | — | 40.4 | — | — |