Synthetic in-context reasoning on MAD synthetic (test)
55.5Compression ScoreFEM-AFT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FEM-AFTbase architecture=AFT, modules=C, L, T, G2026.02 | 55.5 | 9.78 | 90.3 | 80.1 | 90.2 | 93.4 | 69.9 | |
| FEM-SMbase architecture=SMAttn, low-rank convolution (C)=true, LSE mixing (L)=true, linearized temperature (T)=true, outer gate (G)=true2026.02 | 53.1 | 43.1 | 99.9 | 85.9 | 99.9 | 99.3 | 80.2 | |
| FEM-GLAbase architecture=GLA, modules=C, L, T, G, normalization=p~+t(i)2026.02 | 53 | 19.1 | 99.9 | 86.3 | 99.9 | 99 | 74.9 | |
| FEM-MAMBA(-G,T,L,C)base architecture=Mamba2026.02 | 52.7 | 6.7 | 90.4 | 89.5 | 90.1 | 86.3 | 69.3 | |
| FEM-SM(-G)base architecture=SMAttn, low-rank convolution (C)=true, LSE mixing (L)=true, linearized temperature (T)=true, outer gate (G)=false2026.02 | 52.3 | 39.1 | 99.9 | 85.8 | 99.9 | 99.4 | 79.4 | |
| FEM-GLA(-G)base architecture=GLA, modules=C, L, T, normalization=p~+t(i)2026.02 | 51.9 | 13.2 | 97.1 | 86.1 | 93.5 | 91.4 | 72.2 | |
| FEM-SM(-C)base architecture=SMAttn, low-rank convolution (C)=false, LSE mixing (L)=true, linearized temperature (T)=true, outer gate (G)=true2026.02 | 51.2 | 35.4 | 99.9 | 85.9 | 98.5 | 99 | 78.3 | |
| FEM-GLA(-G,T)base architecture=GLA, modules=C, L, normalization=p~+t(i)2026.02 | 51.2 | 12.4 | 92.2 | 85.1 | 92.4 | 89.2 | 70.4 | |
| FEM-MAMBAbase architecture=Mamba, modules=C, L, T, G, normalization=p~+t(i)2026.02 | 51.1 | 16.8 | 90.7 | 89.7 | 92.7 | 97 | 73 | |
| FEM-SM(-C,G)base architecture=SMAttn, low-rank convolution (C)=false, LSE mixing (L)=true, linearized temperature (T)=true, outer gate (G)=false2026.02 | 50.7 | 32.8 | 99.9 | 85.7 | 98 | 97.6 | 77.5 | |
| FEM-MAMBA(-p~t norm)base architecture=Mamba, modules=C, L, T, G, normalization=s+t(i)2026.02 | 50.5 | 12.8 | 93.4 | 88.9 | 86.3 | 92.2 | 70.7 | |
| FEM-AFT(-G,T,L,C)base architecture=AFT2026.02 | 50.5 | 9.15 | 63 | 31.1 | 69.2 | 90.1 | 52.2 | |
| FEM-SM(-G,T)base architecture=SMAttn, low-rank convolution (C)=true, LSE mixing (L)=true, linearized temperature (T)=false, outer gate (G)=false2026.02 | 50.3 | 39 | 99.9 | 85.4 | 99.9 | 98 | 78.8 | |
| FEM-SM(-C,G,T)base architecture=SMAttn, low-rank convolution (C)=false, LSE mixing (L)=true, linearized temperature (T)=false, outer gate (G)=false2026.02 | 49.5 | 26.3 | 99.9 | 85.7 | 97.5 | 97.5 | 76.1 | |
| FEM-GLA(-G,T,L)base architecture=GLA, low-rank convolution (C)=true2026.02 | 47.1 | 9.4 | 91.7 | 83.4 | 92.5 | 88.5 | 68.8 | |
| GatedDeltaNet2026.02 | 45 | 29.8 | 99.9 | 80.2 | 99.9 | 94.3 | 74.9 | |
| FEM-SM(-G,T,L)base architecture=SMAttn, low-rank convolution (C)=true, LSE mixing (L)=false, linearized temperature (T)=false, outer gate (G)=false2026.02 | 45 | 31.4 | 99.9 | 85.5 | 99.9 | 96.3 | 76.3 | |
| Hyena2026.02 | 44.8 | 14.4 | 99 | 89.4 | 98.6 | 93 | 73.2 | |
| FEM-SM(-G,T,L,C)base architecture=SMAttn;Transformer, low-rank convolution (C)=false, LSE mixing (L)=false, linearized temperature (T)=false, outer gate (G)=false2026.02 | 44.3 | 24.5 | 99.9 | 85.7 | 98.5 | 95.1 | 74.7 | |
| Mamba22026.02 | 43.6 | 21.1 | 96.4 | 86.9 | 96.7 | 93.3 | 73 | |
| DiffTrans2026.02 | 42.9 | 39 | 99.9 | 83.7 | 97.1 | 95.8 | 76.4 | |
| MambaArchitecture=two-layer, Optimizer=Muon2026.05 | 42.4 | 6.5 | 75.6 | 87.6 | 71.3 | 95 | 63.1 | |
| DeltaNet2026.02 | 42.2 | 35.7 | 99.9 | 52.8 | 99.9 | 99.9 | 71.7 | |
| FEM-GLA(-G,T,L,C)base architecture=GLA, all modules (C,L,T,G)=false2026.02 | 40.2 | 8.5 | 91.3 | 81.3 | 86.8 | 76.8 | 64.2 | |
| AttnArchitecture=two-layer, Optimizer=Muon2026.05 | 34.2 | 26.8 | 80.3 | 80.7 | 86.1 | 95 | 67.2 | |
| ParallaxArchitecture=two-layer, Optimizer=Muon, Abbreviation=PLX2026.05 | 33.2 | 35.6 | 95.1 | 73.3 | 93.7 | 98.8 | 71.6 | |
| LinAttn2026.02 | 33.1 | 8.2 | 91 | 74.9 | 75.6 | 93.1 | 62.6 | |
| GDNArchitecture=two-layer, Optimizer=Muon2026.05 | 32.5 | 11 | 92 | 79.2 | 90.7 | 93.9 | 66.5 | |
| MesaArchitecture=two-layer, Optimizer=Muon2026.05 | 31 | 21.9 | 99.8 | 86.1 | 99.9 | 56.8 | 65.9 |