List your company

Systematic Generalization on SCAN Around Right (val)

99.8AccuracyEquiLSTM
79.72884.93990.1595.361May 17, 2023

Evaluation results

MethodLinks
EquiLSTMGroup=Dir., Training Iterations=10K, Evaluation Protocol=fine-tuned2023.0599.8
EquiGRUGroup=Dir., Fine-tuning iterations=10K2023.0599.4
LSTMGroup=–, Training Iterations=200K, Evaluation Protocol=trained2023.0598.9
LSTMGroup=–, Zero-shot=true, Iterations=200K2023.0598.9
EquizeroLSTMGroup=Direction, Zero-shot=true, Iterations=200K2023.0598.7
G-LSTMGroup=Dir., Training Iterations=200K, Evaluation Protocol=trained2023.0598.4
EquiZeroLSTMGroup=Dir., Training Iterations=10K, Evaluation Protocol=fine-tuned2023.0598.2
GRUGroup=–, Training iterations=200K2023.0597.9
EquiRNNGroup=Dir., Training iterations=10K iterations fine-tuning2023.0597.7
G-GRUGroup=Dir., Training iterations=200K2023.0597.1
G-RNNGroup=Dir., Training iterations=200K iterations2023.0596.6
RNNGroup=-, Training iterations=200K iterations2023.0594.9
EquiZeroGRUGroup=Dir., Fine-tuning iterations=10K2023.0593.6
EquiZeroRNNGroup=Dir., Training iterations=10K iterations fine-tuning2023.0589.5
EquiLSTMGroup=Direction, Zero-shot=true, Iterations=200K2023.0580.5