Semantic Parsing on GeoQuery Few-shot 32 examples (test)
95.7Program AccuracyGrammar Prompting
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Grammar PromptingDecoding=oracle grammar (G = G[y])2023.05 | 95.7 | 96.1 | |
| Grammar PromptingDecoding=oracle grammar + program constraint2023.05 | 95.7 | 96.8 | |
| Grammar PromptingDecoding=grammar and program constraint (ŷ ∈ L(G))2023.05 | 69.6 | 88.9 | |
| Grammar PromptingDecoding=grammar constraint (Ĝ ⊆ G)2023.05 | 67.9 | 88.6 | |
| Grammar PromptingDecoding=unconstrained decoding2023.05 | 67.1 | 87.5 | |
| Standard PromptingDecoding=constrained decoding (ŷ ∈ L(G))2023.05 | 61.1 | 81.8 | |
| Standard PromptingDecoding=unconstrained decoding2023.05 | 60.7 | 81.5 | |
| Linearized Derivation Tree Prompting2023.05 | 58.6 | 77.5 |