Semantic Parsing on GeoQuery (TMCD)
82.8Exact Match AccCover-LS (Oracle)
Evaluation Results
| Method | Links | |
|---|---|---|
| Cover-LS (Oracle)Model=Codex, Evaluation Protocol=NoFT, Oracle Status=true2022.12 | 82.8 | |
| Cover-LSModel=Codex, Evaluation Protocol=NoFT2022.12 | 79.4 | |
| Cover-UttModel=Codex, Evaluation Protocol=NoFT2022.12 | 77.8 | |
| DPPModel=Codex, Evaluation Protocol=NoFT2022.12 | 77.8 | |
| Top-K (Oracle)Model=Codex, Evaluation Protocol=NoFT, Oracle Status=true2022.12 | 76.2 | |
| T5 (fine tuned w/o prompts)Model=T5-large, Evaluation Protocol=Finetuned2022.12 | 75.4 | |
| Top-KModel=Codex, Evaluation Protocol=NoFT2022.12 | 71.8 | |
| RandomModel=Codex, Evaluation Protocol=NoFT2022.12 | 42 | |
| NQG2024.12 | 41 | |
| Transformer2024.12 | 40 | |
| RU-Transformer2024.12 | 37 | |
| sDTM2024.12 | 36 |