List your company

In-Context Retrieval on RULER (S-NIAH/MK-NIAH Metrics)

100S-NIAH-1 (1K)Mamba-2
99.79299.84699.999.954Jun 25, 2026

Evaluation results

MethodLinks
Mamba-2Architecture=Recurrent2026.061001009755.899.699.662.62159.238.614.42921.221.4
KDAArchitecture=Recurrent2026.0610010099.270.61001008930.677.463.226.25444.228
GDN-2Architecture=Recurrent2026.0610010010097.81001009339.29289.831.872.651.437.8
CARVEArchitecture=Recurrent2026.0610010010098.410010094.84293.691.461.876.855.241.4
Transformer (full attn)Architecture=Hybrid, SWA Ratio=N/A (Full Attention)2026.0610010051.2010010044.2095.894.83775.666.638.2
Mamba-2 + SWAArchitecture=Hybrid (recurrent base + SWA at 3:1 ratio)2026.0610010051.825.410099.652.425.897.886.8488258.639
Gated DeltaNet + SWAArchitecture=Hybrid (recurrent base + SWA at 3:1 ratio)2026.0610010047.222.410099.857.325.694.891.247.29178.444.8
KDA + SWAArchitecture=Hybrid (recurrent base + SWA at 3:1 ratio)2026.0610010051.826.2100100562397.293.451.691.48440.4
Mamba-3 SISO + SWAArchitecture=Hybrid (recurrent base + SWA at 3:1 ratio)2026.0610010049.62610010058.227.89590.44478.865.633.6
Mamba-3 MIMO + SWAArchitecture=Hybrid (recurrent base + SWA at 3:1 ratio)2026.061001004922.81001005327.899.498.454.282.47946.6
GDN-2 + SWAArchitecture=Hybrid (recurrent base + SWA at 3:1 ratio)2026.0610010055.227.410010057.929.299.69955.69384.648
CARVE + SWAArchitecture=Hybrid (recurrent base + SWA at 3:1 ratio)2026.0610010056.828.410010061.430.610010063.295.887.350.6
Gated DeltaNetArchitecture=Recurrent2026.0699.810010097.610010087.23289.854.260.6583727.8