Semantic Parsing on SMCalFlow-CS (16-C)
73.5AccuracyCover-LS (Oracle)
Evaluation Results
| Method | Links | |
|---|---|---|
| Cover-LS (Oracle)Model=Codex, Oracle Status=true2022.12 | 73.5 | |
| T5-base + Tok + Compfew-shot=16-shot, token preprocessing=true, component boundary marking=true2023.05 | 67.7 | |
| T5-base + Tokfew-shot=16-shot, token preprocessing=true2023.05 | 65.1 | |
| T5-basefew-shot=16-shot2023.05 | 63.9 | |
| T5+CSLCSL=true2022.12 | 61.4 | |
| Cover-LSModel=Codex2022.12 | 58.3 | |
| Top-K (Oracle)Model=Codex, Oracle Status=true2022.12 | 54 | |
| Cover-UttModel=Codex2022.12 | 51.3 | |
| BART + R-Dangleshrinterval=6, backbone=BART-large2022.12 | 50.6 | |
| DPPModel=Codex2022.12 | 50 | |
| C2F+SSSS=true2022.12 | 47.4 | |
| COARSE2FINE + SS (Span-level Sup.)few-shot=16-shot, supervision=span-level2023.05 | 47.4 | |
| BARTmodel_size=large2022.12 | 47.2 | |
| BERT2SEQ+SSSS=true2022.12 | 46.8 | |
| T52022.12 | 44.7 | |
| C2F2022.12 | 40.6 | |
| T5 (fine tuned w/o prompts)Model=T5-large2022.12 | 39 | |
| Top-KModel=Codex2022.12 | 35.7 | |
| BERT2SEQ2022.12 | 33.6 | |
| RandomModel=Codex2022.12 | 0.7 |