Semantic Parsing on COGS
99.5AccuracyT5 + CSL-Aug*
Evaluation results
| Method | Links | |
|---|---|---|
| T5 + CSL-Aug*Pre-training=Language Pre-training, Augmentation=CSL-Aug*2022.01 | 99.5 | |
| PiSSA-r3Params=3.27M, Best Ep=5.7, Seeds=32026.06 | 98.6 | |
| DoRA-r3Params=3.84M, Best Ep=5.7, Seeds=32026.06 | 98 | |
| LeARPre-training=None2022.01 | 97.7 | |
| LoRA-r64Params=69.73M, Best Ep=5.6, Seeds=52026.06 | 97.64 | |
| LoRA-r3Params=3.27M, Best Ep=6.3, Seeds=32026.06 | 97.53 | |
| DoRA-r16Params=34.09M, Best Ep=10.0, Seeds=32026.06 | 96.9 | |
| LoRA-r16 replay10Params=17.43M, Best Ep=10.0, Seeds=52026.06 | 95.84 | |
| LoRA-r16 replay1Params=17.43M, Best Ep=9.6, Seeds=52026.06 | 95.52 | |
| LoRA-r16Params=17.43M, Best Ep=9.6, Seeds=52026.06 | 95.44 | |
| LoRA-r16 replay5Params=17.43M, Best Ep=9.0, Seeds=52026.06 | 95.4 | |
| FProj-LRParams=17.43M, Best Ep=10.8, Seeds=32026.06 | 94.6 | |
| Fora-LRParams=17.76M, Best Ep=11.0, Seeds=32026.06 | 94.48 | |
| OPLoRA-r16Params=17.43M, Best Ep=10.8, Seeds=52026.06 | 94.44 | |
| CorDA-k16-r16Params=17.43M, Best Ep=8.8, Seeds=52026.06 | 94.16 | |
| T5 + LEXSYMPre-training=Language Pre-training, Augmentation=LEXSYM2022.01 | 83.62 | |
| T5Pre-training=Language Pre-training2022.01 | 83.3 | |
| LexLSTMPre-training=None2022.01 | 82.17 | |
| LSTM + SubstitutePre-training=None, Augmentation=Substitute2022.01 | 81.99 | |
| LSTM + LEXSYMPre-training=None, Augmentation=LEXSYM2022.01 | 81.86 | |
| LSTMPre-training=None2022.01 | 61.13 | |
| GECAPre-training=None2022.01 | 47.74 |