List your company

Systematic Generalization on SCAN Around Right (test)

95.7AccuracyEquiLSTM
-3.72422.08847.973.712May 17, 2023

Evaluation results

MethodLinks
EquiLSTMGroup=Dir., Training Iterations=10K, Evaluation Protocol=fine-tuned2023.0595.7
EquiZeroLSTMGroup=Dir., Training Iterations=10K, Evaluation Protocol=fine-tuned2023.0592.5
EquiGRUGroup=Dir., Fine-tuning iterations=10K2023.0591.6
G-LSTMGroup=Dir., Training Iterations=200K, Evaluation Protocol=trained2023.0589.6
G-RNNGroup=Dir., Training iterations=200K iterations2023.0584.5
G-GRUGroup=Dir., Training iterations=200K2023.0582.7
EquizeroLSTMGroup=Direction, Zero-shot=true, Iterations=200K2023.0581.7
EquiRNNGroup=Dir., Training iterations=10K iterations fine-tuning2023.0578.4
EquiZeroGRUGroup=Dir., Fine-tuning iterations=10K2023.0574.6
EquiZeroRNNGroup=Dir., Training iterations=10K iterations fine-tuning2023.0564.3
EquiLSTMGroup=Direction, Zero-shot=true, Iterations=200K2023.0528.2
RNNGroup=-, Training iterations=200K iterations2023.055.9
LSTMGroup=–, Training Iterations=200K, Evaluation Protocol=trained2023.050.4
LSTMGroup=–, Zero-shot=true, Iterations=200K2023.050.4
GRUGroup=–, Training iterations=200K2023.050.1