Systematic Generalization on SCAN Add Jump (val)
99.6AccuracyG-GRU
Evaluation Results
| Method | Links | |
|---|---|---|
| G-GRUGroup=Verb, Training iterations=200K2023.05 | 99.6 | |
| G-LSTMGroup=Verb, Training Iterations=200K, Evaluation Protocol=trained2023.05 | 99.4 | |
| LSTMGroup=–, Training Iterations=200K, Evaluation Protocol=trained2023.05 | 99.1 | |
| LSTMGroup=–, Zero-shot=true, Iterations=200K2023.05 | 99.1 | |
| EquiLSTMGroup=Verb, Training Iterations=10K, Evaluation Protocol=fine-tuned2023.05 | 98.9 | |
| EquizeroLSTMGroup=Verb, Zero-shot=true, Iterations=200K2023.05 | 98.4 | |
| EquiZeroLSTMGroup=Verb, Training Iterations=10K, Evaluation Protocol=fine-tuned2023.05 | 98.3 | |
| GRUGroup=–, Training iterations=200K2023.05 | 96.9 | |
| EquiZeroGRUGroup=Verb, Fine-tuning iterations=10K2023.05 | 96.4 | |
| EquiGRUGroup=Verb, Fine-tuning iterations=10K2023.05 | 95.7 | |
| G-RNNGroup=Verb, Training iterations=200K iterations2023.05 | 93.2 | |
| EquiRNNGroup=Verb, Training iterations=10K iterations fine-tuning2023.05 | 92.2 | |
| EquiZeroRNNGroup=Verb, Training iterations=10K iterations fine-tuning2023.05 | 91.7 | |
| RNNGroup=-, Training iterations=200K iterations2023.05 | 91.4 | |
| EquiLSTMGroup=Verb, Zero-shot=true, Iterations=200K2023.05 | 62.5 |