Instruction Following on SCAN around right
99.51AccuracyLSTM + LEXSYM
Evaluation results
| Method | Links | |
|---|---|---|
| LSTM + LEXSYMPre-training=None, Augmentation=LEXSYM2022.01 | 99.51 | |
| LSTM + SubstitutePre-training=None, Augmentation=Substitute2022.01 | 99.17 | |
| GECAPre-training=None2022.01 | 98.5 | |
| T5 + LEXSYMPre-training=Language Pre-training, Augmentation=LEXSYM2022.01 | 97.29 | |
| LexLSTMPre-training=None2022.01 | 88.41 | |
| T5Pre-training=Language Pre-training2022.01 | 38.4 | |
| LSTMPre-training=None2022.01 | 8.65 |