Semantic Parsing on Overnight Blk Few-shot 32 examples (test)
74.4Program AccGrammar Prompting
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Grammar PromptingDecoding=oracle grammar + program constraint2023.05 | 74.4 | 96.5 | |
| Grammar PromptingDecoding=oracle grammar (G = G[y])2023.05 | 73.9 | 94.2 | |
| Grammar PromptingDecoding=grammar and program constraint (ŷ ∈ L(G))2023.05 | 37.6 | 60.9 | |
| Grammar PromptingDecoding=grammar constraint (Ĝ ⊆ G)2023.05 | 37.1 | 60.4 | |
| Grammar PromptingDecoding=unconstrained decoding2023.05 | 34.8 | 57.4 | |
| Standard PromptingDecoding=unconstrained decoding2023.05 | 29.3 | 54.7 | |
| Standard PromptingDecoding=constrained decoding (ŷ ∈ L(G))2023.05 | 29.3 | 54.7 | |
| Linearized Derivation Tree Prompting2023.05 | 27.3 | 56.4 |