Information Extraction on SWDE
92.1AccuracyNemotron-Nano 12B v2
Evaluation results
| Method | Links | |
|---|---|---|
| Nemotron-Nano 12B v2Speedup @32k=5.85x, Speedup @16k=4.3x2026.04 | 92.1 | |
| Nemotron-3-Nano 30BSpeedup @32k=4.09x, Speedup @16k=2.8x2026.04 | 91.7 | |
| Qwen-3.5 27BSpeedup @32k=0.55x, Speedup @16k=0.5x2026.04 | 90.4 | |
| Apriel-1.6Speedup @32k=1.0x, Speedup @16k=1.0x2026.04 | 90.2 | |
| Reg|Lklhd–18Speedup @32k=4.76x, Speedup @16k=2.2x2026.04 | 89.8 | |
| Falcon-H1R 7BSpeedup @32k=4.61x, Speedup @16k=3.4x2026.04 | 89.6 | |
| all-attentionSpeedup @32k=1.0x, Speedup @16k=1.0x2026.04 | 89.5 | |
| OLMo-Hybrid-Think 7BSpeedup @32k=2.51x, Speedup @16k=2.1x2026.04 | 89.5 | |
| Idealized|Lklhd–6Speedup @32k=6.2x, Speedup @16k=2.4x2026.04 | 88.9 | |
| Apriel-H1 15BSpeedup @32k=1.97x, Speedup @16k=1.9x2026.04 | 88.6 | |
| Reg|Lklhd–26Speedup @32k=2.85x, Speedup @16k=1.5x2026.04 | 88.2 | |
| S1: Distil. Idealized|All–6Speedup @32k=6.13x, Speedup @16k=2.5x2026.04 | 87.8 | |
| Idealized|All–6Speedup @32k=6.13x, Speedup @16k=2.5x2026.04 | 87.4 | |
| Idealized|All–18Speedup @32k=1.99x, Speedup @16k=1.1x2026.04 | 86.6 | |
| Reg|Lklhd–13Speedup @32k=6.9x, Speedup @16k=2.7x2026.04 | 85.1 | |
| Reg|Lklhd–10Speedup @32k=10.69x, Speedup @16k=4.2x2026.04 | 82.9 | |
| Mistral (Full-Attention)Parameters=790M, Zero-shot=true2024.07 | 61.2 | |
| Mistral (Full-Attention)Parameters=370M, Zero-shot=true2024.07 | 47.16 | |
| BMoJo (Fading + Eidetic)Parameters=790M, Zero-shot=true2024.07 | 23.4 | |
| BMoJo (Fading)Parameters=790M, Zero-shot=true2024.07 | 22.59 | |
| BMoJo (Fading + Eidetic)Parameters=370M, Zero-shot=true2024.07 | 17.91 | |
| Mamba (SSM)Parameters=790M, Zero-shot=true2024.07 | 17.37 | |
| BMoJo (Fading)Parameters=370M, Zero-shot=true2024.07 | 15.56 | |
| Hybrid (Sliding Attention + SSM)Parameters=370M, Zero-shot=true2024.07 | 8.55 | |
| Hybrid (Sliding Attention + SSM)Parameters=790M, Zero-shot=true2024.07 | 8.37 | |
| Mamba (SSM)Parameters=370M, Zero-shot=true2024.07 | 7.38 | |
| Memory Transformer (Combined objective)Objective=Combined, dm=512, nl=16, nctx=256, s=42026.02 | 0.81 | |
| Memory Transformer (Causal objective)Objective=Causal, dm=512, nl=16, nctx=256, s=42026.02 | 0 |