Semantic Parsing on CFQ (MCD1)
98.6AccuracyGrounded Graph Decoding
Evaluation Results
| Method | Links | |
|---|---|---|
| Grounded Graph DecodingAlgorithm Type=CFQ-specific2023.05 | 98.6 | |
| Dynamic Least-to-Most PromptingAlgorithm Type=CFQ-specific2023.05 | 94.3 | |
| LEARAlgorithm Type=CFQ-specific2023.05 | 91.7 | |
| LSPBackbone=T5-base, Algorithm Type=Domain general2023.05 | 88.7 | |
| LIR+RIRBackbone=T5-3B, Algorithm Type=CFQ-specific2023.05 | 88.4 | |
| LIR+RIRBackbone=T5-base, Algorithm Type=CFQ-specific2023.05 | 85.8 | |
| Hierarchical Poset DecodingHierarchical mechanism status=enabled2020.10 | 79.6 | |
| HPDAlgorithm Type=CFQ-specific2023.05 | 79.6 | |
| Hierarchical Poset DecodingSketch prediction method=Seq2Tree-based2020.10 | 75.7 | |
| Hierarchical Poset DecodingSketch prediction method=Seq2Seq-based2020.10 | 74.3 | |
| LSPBackbone=LSTM Encoder, Algorithm Type=Domain general2023.05 | 73 | |
| T5Backbone=T5-3B, Algorithm Type=Domain general2023.05 | 65 | |
| T5-11B-modAlgorithm Type=CFQ-specific2023.05 | 61.6 | |
| T5Backbone=T5-base, Algorithm Type=Domain general2023.05 | 58.5 | |
| LAGRAlgorithm Type=CFQ-specific2023.05 | 57.9 | |
| TransformerSPARQL expression type=simplified2020.10 | 53 | |
| Edge Transformer2021.12 | 47.73 | |
| Edge TransformerAlgorithm Type=Domain general2023.05 | 47.7 | |
| Universal Transformer2021.12 | 42.7 | |
| T5-smallsource_paper=Furrer et al. (2020)2021.12 | 42.5 | |
| Evolved Transformersource_paper=Furrer et al. (2020)2021.12 | 42.4 | |
| Evolved TransformerAlgorithm Type=Domain general2023.05 | 42.4 | |
| LSTM+AttentionSPARQL expression type=simplified2020.10 | 42.2 | |
| Universal TransformerSPARQL expression type=standard2020.10 | 37.4 | |
| Universal Transformersource_paper=Keysers et al. (2020)2021.12 | 37.4 | |
| Universal TransformerAlgorithm Type=Domain general2023.05 | 37.4 | |
| TransformerSPARQL expression type=standard2020.10 | 34.9 | |
| TransformerAlgorithm Type=Domain general2023.05 | 34.9 | |
| LSTM+AttentionSPARQL expression type=standard2020.10 | 28.9 | |
| LSTMAlgorithm Type=Domain general2023.05 | 28.9 | |
| Seq2Tree2020.10 | 24.3 | |
| Hierarchical Poset DecodingHierarchical mechanism status=disabled2020.10 | 21.3 | |
| CGPS2020.10 | 4.81 |