Semantic Parsing on WIKITABLEQUESTIONS (test)
52.3Execution Accuracy (Best)TaBERT-Large
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TaBERT-Largebackbone=TaBERT-Large, K (number of rows)=3, content_snapshot=included, base_system=MAPO2020.05 | 52.3 | — | 51.8 | 0.6 | |
| TaBERT-Largebackbone=TaBERT-Large, K (number of rows)=1, content_snapshot=included, base_system=MAPO2020.05 | 51.5 | — | 51.2 | 0.9 | |
| TaBERT-Basebackbone=TaBERT-Base, K (number of rows)=3, content_snapshot=included, base_system=MAPO2020.05 | 51.3 | — | 51.4 | 0.3 | |
| TaBERT-Basebackbone=TaBERT-Base, K (number of rows)=1, content_snapshot=included, base_system=MAPO2020.05 | 51.2 | — | 50.4 | 0.5 | |
| MAPO w/ BERT-Basebackbone=BERT-Base, K (number of rows)=1, content_snapshot=excluded2020.05 | 50.2 | — | 48.8 | 0.9 | |
| MAPO w/ BERT-Largebackbone=BERT-Large, K (number of rows)=1, content_snapshot=included2020.05 | 50.1 | — | 49.6 | 0.5 | |
| TaBERT-Basebackbone=TaBERT-Base, K (number of rows)=1, content_snapshot=excluded, base_system=MAPO2020.05 | 50 | — | 49.4 | 0.4 | |
| MAPO w/ BERT-Basebackbone=BERT-Base, K (number of rows)=1, content_snapshot=included2020.05 | 49.2 | — | 49.4 | 0.5 | |
| Ensemble 10 ModelsEnsemble size=102020.05 | 46.9 | — | — | — | |
| Wang et al. (2019b)2020.05 | 44.5 | — | — | — | |
| Dasigi et al. (2019)2020.05 | 44.3 | — | — | — | |
| Agarwal et al. (2019)2020.05 | 44.1 | — | — | — | |
| Base Parser (MAPO)base_system=MAPO2020.05 | 43.8 | — | 43.1 | 0.5 | |
| Zhang et al. (2017)2020.05 | 43.7 | — | — | — | |
| Ensemble 15 ModelsEnsemble size=152020.05 | 37.7 | — | — | — | |
| Pasupat and Liang (2015)2020.05 | 37.1 | — | — | — | |
| Neelakantan et al. (2016)2020.05 | 34.2 | — | — | — | |
| Abstract Programs f.w. standard attentionSupervision=Denotations, Attention=standard2019.09 | — | 41.4 | — | — | |
| Agarwal et al. (2019)Supervision=Denotations2019.09 | — | 44.1 | — | — | |
| Dasigi et al. (2019)Supervision=Denotations2019.09 | — | 43.9 | — | — | |
| Haug et al. (2018)Supervision=Denotations2019.09 | — | 34.8 | — | — | |
| Liang et al. (2018)Supervision=Denotations2019.09 | — | 43.1 | — | — | |
| Neelakantan et al. (2017)Supervision=Denotations2019.09 | — | 34.2 | — | — | |
| Pasupat and Liang (2015)Supervision=Denotations2019.09 | — | 37.1 | — | — | |
| Structured-Attention ParserSupervision=Denotations, Attention=structured2019.09 | — | 44.5 | — | — | |
| Typed Seq2SeqSupervision=Denotations2019.09 | — | 38.3 | — | — | |
| Zhang et al. (2017)Supervision=Denotations2019.09 | — | 43.7 | — | — |