List your company
ResearchFormerly WizWand

Semantic Parsing on GeoQuery (Templ.)

85.9Exact Match AccuracyT5 (fine tuned w/o prompts)
48.25258.02667.877.574Dec 13, 2022
Updated 8mo ago

Evaluation results

MethodLinks
T5 (fine tuned w/o prompts)Model=T5-large, Evaluation Protocol=Finetuned, Oracle Status=false2022.1285.9
Cover-LSModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.1285.3
Cover-UttModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.1282.1
DPPModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.1281.2
Cover-LS (Oracle)Model=Codex, Evaluation Protocol=NoFT, Oracle Status=true2022.1281.2
Top-KModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.1278
Top-K (Oracle)Model=Codex, Evaluation Protocol=NoFT, Oracle Status=true2022.1274.5
RandomModel=Codex, Evaluation Protocol=NoFT, Oracle Status=false2022.1249.7