Instruction Following on SCAN jump
100AccuracyLSTM + LEXSYM
Evaluation results
| Method | Links | |
|---|---|---|
| LSTM + LEXSYMPre-training=None, Augmentation=LEXSYM2022.01 | 100 | |
| T5 + LEXSYMPre-training=Language Pre-training, Augmentation=LEXSYM2022.01 | 99.96 | |
| LSTM + SubstitutePre-training=None, Augmentation=Substitute2022.01 | 99.95 | |
| GECAPre-training=None2022.01 | 99.94 | |
| T5 + CSL-Aug*Pre-training=Language Pre-training, Augmentation=CSL-Aug*2022.01 | 99.7 | |
| LexLSTMPre-training=None2022.01 | 99.14 | |
| PModelconstraint=hard constraint d(r) < inf2023.06 | 97.08 | |
| vNQ¹constraint=hard constraint d(r) < inf2023.06 | 96.8 | |
| T5Pre-training=Language Pre-training2022.01 | 93.6 | |
| G2L22022.02 | 1 | |
| SwitchOutapplied after GECA=true2022.02 | 0.98 | |
| SeqMixapplied after GECA=true2022.02 | 0.98 | |
| recomb-22022.02 | 0.88 | |
| GECA2022.02 | 0.87 | |
| WordDropapplied after GECA=true2022.02 | 0.52 | |
| LSTMPre-training=None2022.01 | 0.41 | |
| seq2seq2022.02 | 0 | |
| Transformer2022.02 | 0 |