Semantic Parsing on GeoQuery (Templ.)
85.9Exact Match AccuracyT5 (fine tuned w/o prompts)
Evaluation Results
| Method | Links | |
|---|---|---|
| T5 (fine tuned w/o prompts)Model=T5-large, Evaluation Protocol=Finetuned, Oracle Status=false2022.12 | 85.9 | |
| Cover-LSModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.12 | 85.3 | |
| Cover-UttModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.12 | 82.1 | |
| DPPModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.12 | 81.2 | |
| Cover-LS (Oracle)Model=Codex, Evaluation Protocol=NoFT, Oracle Status=true2022.12 | 81.2 | |
| Top-KModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.12 | 78 | |
| Top-K (Oracle)Model=Codex, Evaluation Protocol=NoFT, Oracle Status=true2022.12 | 74.5 | |
| RandomModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.12 | 49.7 |