Semantic Parsing on CFQ (MCD2)
95.3AccuracyDynamic Least-to-Most Prompting
Evaluation results
| Method | Links | |
|---|---|---|
| Dynamic Least-to-Most PromptingAlgorithm Type=CFQ-specific2023.05 | 95.3 | |
| LEARAlgorithm Type=CFQ-specific2023.05 | 89.2 | |
| LIR+RIRBackbone=T5-3B, Algorithm Type=CFQ-specific2023.05 | 85.3 | |
| Grounded Graph DecodingAlgorithm Type=CFQ-specific2023.05 | 67.9 | |
| LIR+RIRBackbone=T5-base, Algorithm Type=CFQ-specific2023.05 | 64 | |
| Hierarchical Poset DecodingHierarchical mechanism status=enabled2020.10 | 59.6 | |
| HPDAlgorithm Type=CFQ-specific2023.05 | 59.6 | |
| LSPBackbone=T5-base, Algorithm Type=Domain general2023.05 | 57.7 | |
| Hierarchical Poset DecodingSketch prediction method=Seq2Seq-based2020.10 | 45.7 | |
| T5Backbone=T5-3B, Algorithm Type=Domain general2023.05 | 41 | |
| Hierarchical Poset DecodingSketch prediction method=Seq2Tree-based2020.10 | 40.9 | |
| T5-11B-modAlgorithm Type=CFQ-specific2023.05 | 31.3 | |
| LSPBackbone=LSTM Encoder, Algorithm Type=Domain general2023.05 | 29.5 | |
| T5Backbone=T5-base, Algorithm Type=Domain general2023.05 | 27 | |
| LAGRAlgorithm Type=CFQ-specific2023.05 | 26 | |
| TransformerSPARQL expression type=simplified2020.10 | 19.5 | |
| LSTM+AttentionSPARQL expression type=simplified2020.10 | 14.5 | |
| Edge Transformer2021.12 | 13.14 | |
| Edge TransformerAlgorithm Type=Domain general2023.05 | 13.1 | |
| T5-smallsource_paper=Furrer et al. (2020)2021.12 | 11.2 | |
| Universal Transformer2021.12 | 9.46 | |
| Evolved Transformersource_paper=Furrer et al. (2020)2021.12 | 9.3 | |
| Evolved TransformerAlgorithm Type=Domain general2023.05 | 9.3 | |
| TransformerSPARQL expression type=standard2020.10 | 8.2 | |
| TransformerAlgorithm Type=Domain general2023.05 | 8.2 | |
| Universal TransformerSPARQL expression type=standard2020.10 | 8.1 | |
| Universal Transformersource_paper=Keysers et al. (2020)2021.12 | 8.1 | |
| Universal TransformerAlgorithm Type=Domain general2023.05 | 8.1 | |
| Hierarchical Poset DecodingHierarchical mechanism status=disabled2020.10 | 6.4 | |
| LSTM+AttentionSPARQL expression type=standard2020.10 | 5 | |
| LSTMAlgorithm Type=Domain general2023.05 | 5 | |
| Seq2Tree2020.10 | 4.1 | |
| CGPS2020.10 | 1.04 |