Semantic Parsing on SMCalFlow Few-shot 16 examples (test)
83.6Program AccuracyGrammar Prompting
Evaluation results
| Method | Links | |
|---|---|---|
| Grammar PromptingDecoding=oracle grammar + program constraint2023.05 | 83.6 | |
| Grammar PromptingDecoding=oracle grammar (G = G[y])2023.05 | 80 | |
| Grammar PromptingDecoding=grammar and program constraint (ŷ ∈ L(G))2023.05 | 52.4 | |
| Grammar PromptingDecoding=grammar constraint (Ĝ ⊆ G)2023.05 | 51.3 | |
| Grammar PromptingDecoding=unconstrained decoding2023.05 | 50.8 | |
| Linearized Derivation Tree Prompting2023.05 | 50 | |
| Standard PromptingDecoding=constrained decoding (ŷ ∈ L(G))2023.05 | 49.2 | |
| Standard PromptingDecoding=unconstrained decoding2023.05 | 46.4 |