List your company
ResearchFormerly WizWand

Semantic Parsing on GeoQuery (TMCD)

82.8Exact Match AccCover-LS (Oracle)
34.12846.76459.472.036Dec 13, 2022Apr 14, 2023Aug 15, 2023Dec 16, 2023Apr 16, 2024Aug 17, 2024Dec 18, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Cover-LS (Oracle)Model=Codex, Evaluation Protocol=NoFT, Oracle Status=true2022.1282.8
Cover-LSModel=Codex, Evaluation Protocol=NoFT2022.1279.4
Cover-UttModel=Codex, Evaluation Protocol=NoFT2022.1277.8
DPPModel=Codex, Evaluation Protocol=NoFT2022.1277.8
Top-K (Oracle)Model=Codex, Evaluation Protocol=NoFT, Oracle Status=true2022.1276.2
T5 (fine tuned w/o prompts)Model=T5-large, Evaluation Protocol=Finetuned2022.1275.4
Top-KModel=Codex, Evaluation Protocol=NoFT2022.1271.8
RandomModel=Codex, Evaluation Protocol=NoFT2022.1242
NQG2024.1241
Transformer2024.1240
RU-Transformer2024.1237
sDTM2024.1236