Semantic Parsing on CFQ MCD avg
95Exact Match AccuracyDynamic Least-to-Most Prompting
Evaluation Results
| Method | Links | |
|---|---|---|
| Dynamic Least-to-Most PromptingAlgorithm Type=CFQ-specific2023.05 | 95 | |
| LEARAlgorithm Type=CFQ-specific2023.05 | 90.9 | |
| LIR+RIRBackbone=T5-3B, Algorithm Type=CFQ-specific2023.05 | 83.8 | |
| Grounded Graph DecodingAlgorithm Type=CFQ-specific2023.05 | 81.3 | |
| HPDAlgorithm Type=CFQ-specific2023.05 | 69 | |
| LIR+RIRBackbone=T5-base, Algorithm Type=CFQ-specific2023.05 | 67.8 | |
| LSPBackbone=T5-base, Algorithm Type=Domain general2023.05 | 63.4 | |
| T5Backbone=T5-3B, Algorithm Type=Domain general2023.05 | 49.5 | |
| T5-11B-modAlgorithm Type=CFQ-specific2023.05 | 42.1 | |
| LSPBackbone=LSTM Encoder, Algorithm Type=Domain general2023.05 | 41.9 | |
| LAGRAlgorithm Type=CFQ-specific2023.05 | 34.9 | |
| T5Backbone=T5-base, Algorithm Type=Domain general2023.05 | 34.6 | |
| Edge TransformerAlgorithm Type=Domain general2023.05 | 24.7 | |
| Edge Transformer2021.12 | 24.69 | |
| T5-smallsource_paper=Furrer et al. (2020)2021.12 | 21.4 | |
| Universal Transformer2021.12 | 21.26 | |
| Evolved Transformersource_paper=Furrer et al. (2020)2021.12 | 20.8 | |
| Evolved TransformerAlgorithm Type=Domain general2023.05 | 20.8 | |
| Universal Transformersource_paper=Keysers et al. (2020)2021.12 | 18.9 | |
| Universal TransformerAlgorithm Type=Domain general2023.05 | 18.9 | |
| TransformerAlgorithm Type=Domain general2023.05 | 17.8 | |
| LSTMAlgorithm Type=Domain general2023.05 | 14.9 |