Semantic Parsing on SMCalFlow-CS (0-C)
40.7Exact Match AccCover-LS (Oracle)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Cover-LS (Oracle)Model=Codex, Oracle Status=true2022.12 | 40.7 | — | — | |
| Cover-UttModel=Codex2022.12 | 35.3 | — | — | |
| DPPModel=Codex2022.12 | 34.7 | — | — | |
| Top-KModel=Codex2022.12 | 17 | — | — | |
| RandomModel=Codex2022.12 | 1.3 | — | — | |
| Cover-LSModel=Codex2022.12 | 0.3 | — | — | |
| T5 (fine tuned w/o prompts)Model=T5-large2022.12 | 0 | — | — | |
| Top-K (Oracle)Model=Codex, Oracle Status=true2022.12 | 0 | — | — | |
| Cover-LS (NoFT)2022.12 | — | 0 | 0 | |
| PaLM 540B (ICL)Source=Qiu et al. 2022b2022.12 | — | — | 4.7 | |
| PaLM 62B (FT)Source=Qiu et al. 2022b2022.12 | — | — | 26.9 | |
| T5 11B (Prompt Tuning)Source=Qiu et al. 2022b2022.12 | — | — | 0 | |
| T5 Base (FT)Source=Qiu et al. 2022b2022.12 | — | — | 21.7 | |
| T5 Base + CSL-Aug (FT)Source=Qiu et al. 2022a2022.12 | — | 34.7 | — | |
| T5 Large (fine tuned w/o prompts)2022.12 | — | 0 | 0 | |
| Top-K (NoFT)2022.12 | — | 19.8 | 13.6 |