Semantic Parsing on COVR-10 (test)
83.2Exact Match AccuracyCover-LS (Oracle)
Evaluation Results
| Method | Links | |
|---|---|---|
| Cover-LS (Oracle)Model=Codex, Oracle Status=true2022.12 | 83.2 | |
| Cover-UttModel=Codex2022.12 | 78.1 | |
| RandomModel=Codex2022.12 | 69.4 | |
| Cover-LSModel=Codex2022.12 | 64.4 | |
| DPPModel=Codex2022.12 | 62.7 | |
| Top-KModel=Codex2022.12 | 61.8 | |
| Top-K (Oracle)Model=Codex, Oracle Status=true2022.12 | 35.4 | |
| T5 (fine tuned w/o prompts)Model=T5-large2022.12 | 21.5 |