Semantic Parsing on Spider (dev)
75.5Exact Match AccuracyPICARD
Evaluation results
| Method | Links | |
|---|---|---|
| PICARDExtra Pre-training=false2022.01 | 75.5 | |
| SmBopExtra Pre-training=true2022.01 | 74.7 | |
| T5-3BMode=finetune2022.01 | 71.76 | |
| T5-3BBackbone=T5-3B, Extra Pre-training=false2022.01 | 71.76 | |
| T5-largeBackbone=T5-large, Extra Pre-training=false2022.01 | 66.63 | |
| T5-baseBackbone=T5-base, Extra Pre-training=false2022.01 | 58.12 | |
| Codex 175BFew-shot selection strategy=random2022.01 | 43.23 | |
| Codex 175BFew-shot selection strategy=select2022.01 | 40.72 | |
| GPT-3 175BFew-shot selection strategy=select2022.01 | 20 | |
| GPT-3 175BFew-shot selection strategy=random2022.01 | 18.33 | |
| T0 3BMode=zero-shot2022.01 | 0 |