Semantic Parsing on GeoQuery compositional
89.5AccuracySpanSub
Evaluation results
| Method | Links | |
|---|---|---|
| SpanSubBase Model=BART2023.06 | 89.5 | |
| T5 Base + CSL-Aug (FT)Source=Qiu et al. 2022a2022.12 | 89.3 | |
| SUBSBase Model=BART2023.06 | 88.3 | |
| T5 11B (Prompt Tuning)Source=Qiu et al. 2022b2022.12 | 87.7 | |
| LexSymBase Model=BART2023.06 | 87.7 | |
| PaLM 62B (FT)Source=Qiu et al. 2022b2022.12 | 85.1 | |
| T5 Base (FT)Source=Qiu et al. 2022a2022.12 | 84.8 | |
| T5 Base (FT)Source=Qiu et al. 2022b2022.12 | 84.8 | |
| T5 Large (fine tuned w/o prompts)2022.12 | 83.8 | |
| GECABase Model=BART2023.06 | 83 | |
| Cover-LS (NoFT)2022.12 | 81.6 | |
| SpanSubBase Model=LSTM2023.06 | 81.4 | |
| T5 11B (Prompt Tuning)Source=Qiu et al. 2022b2022.12 | 81.2 | |
| LexSymBase Model=LSTM2023.06 | 80.2 | |
| SUBSBase Model=LSTM2023.06 | 77.7 | |
| PaLM 540B (ICL)Source=Qiu et al. 2022b2022.12 | 76.6 | |
| Cover-LS (NoFT)2022.12 | 76.3 | |
| SpanParse2023.06 | 76.3 | |
| T5 Base + CSL-Aug (FT)Source=Qiu et al. 2022a2022.12 | 74.9 | |
| Top-K (NoFT)2022.12 | 74.7 | |
| T5 Large (fine tuned w/o prompts)2022.12 | 73.5 | |
| PaLM 62B (FT)Source=Qiu et al. 2022b2022.12 | 72.7 | |
| BARTBase Model=BART2023.06 | 71.9 | |
| Top-K (NoFT)2022.12 | 69.4 | |
| T5 Base (FT)Source=Qiu et al. 2022a2022.12 | 69.2 | |
| T5 Base (FT)Source=Qiu et al. 2022b2022.12 | 69.2 | |
| PaLM 540B (ICL)Source=Qiu et al. 2022b2022.12 | 63.6 | |
| GECABase Model=LSTM2023.06 | 60.6 | |
| LSTMBase Model=LSTM2023.06 | 58.6 |