Systematic Generalization on SCAN Add Jump (test)
99.8AccuracyG-GRU
Evaluation Results
| Method | Links | |
|---|---|---|
| G-GRUGroup=Verb, Training iterations=200K2023.05 | 99.8 | |
| G-LSTMGroup=Verb, Training Iterations=200K, Evaluation Protocol=trained2023.05 | 98.3 | |
| EquiLSTMGroup=Verb, Training Iterations=10K, Evaluation Protocol=fine-tuned2023.05 | 97.9 | |
| EquiZeroLSTMGroup=Verb, Training Iterations=10K, Evaluation Protocol=fine-tuned2023.05 | 97.9 | |
| EquiZeroGRUGroup=Verb, Fine-tuning iterations=10K2023.05 | 93.6 | |
| G-RNNGroup=Verb, Training iterations=200K iterations2023.05 | 87.4 | |
| EquiZeroRNNGroup=Verb, Training iterations=10K iterations fine-tuning2023.05 | 84.8 | |
| EquiRNNGroup=Verb, Training iterations=10K iterations fine-tuning2023.05 | 83.9 | |
| EquiGRUGroup=Verb, Fine-tuning iterations=10K2023.05 | 81.1 | |
| EquizeroLSTMGroup=Verb, Zero-shot=true, Iterations=200K2023.05 | 75.2 | |
| EquiLSTMGroup=Verb, Zero-shot=true, Iterations=200K2023.05 | 7.1 | |
| RNNGroup=-, Training iterations=200K iterations2023.05 | 0.2 | |
| LSTMGroup=–, Training Iterations=200K, Evaluation Protocol=trained2023.05 | 0 | |
| GRUGroup=–, Training iterations=200K2023.05 | 0 | |
| LSTMGroup=–, Zero-shot=true, Iterations=200K2023.05 | 0 |