List your company
ResearchFormerly WizWand

Language Modeling on WikiText and LAMBADA (test)

7.3WikiText PPLLlama3 8B
-64.408419.621903.651,387.679Nov 18, 2025Dec 2, 2025Dec 16, 2025Dec 30, 2025Jan 13, 2026Jan 27, 2026Feb 11, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Llama3 8BCR=uncompressed, Backbone=Llama3-8B2026.027.33.1
ROCKETCR=0.2, Backbone=Llama3-8B2026.02113.8
DNH-HOPEModel Parameters=1.3B, Training Tokens=100B, Evaluation Protocol=Static setting2025.1114.9210.87
ROCKETCR=0.3, Backbone=Llama3-8B2026.02154.4
Titans (LMM)Model Parameters=1.3B, Training Tokens=100B, Evaluation Protocol=Static setting2025.1115.611.41
DeltaNetModel Parameters=1.3B, Training Tokens=100B, Evaluation Protocol=Static setting2025.1117.7116.88
Transformer++Model Parameters=1.3B, Training Tokens=100B, Evaluation Protocol=Static setting2025.1118.5318.32
RetNetModel Parameters=1.3B, Training Tokens=100B, Evaluation Protocol=Static setting2025.1119.0817.27
DNH-HOPEModel Parameters=760M, Training Tokens=30B, Evaluation Protocol=Static setting2025.1119.8220.15
CoSpaDiCR=0.2, Backbone=Llama3-8B2026.02204.3
Titans (LMM)Model Parameters=760M, Training Tokens=30B, Evaluation Protocol=Static setting2025.1120.0421.96
HOPEModel Parameters=1.3B, Training Tokens=100B, Evaluation Protocol=Static setting2025.1120.5320.47
DeltaNetModel Parameters=760M, Training Tokens=30B, Evaluation Protocol=Static setting2025.1124.3724.6
Transformer++Model Parameters=760M, Training Tokens=30B, Evaluation Protocol=Static setting2025.1125.2127.64
HOPEModel Parameters=760M, Training Tokens=30B, Evaluation Protocol=Static setting2025.1126.0529.38
RetNetModel Parameters=760M, Training Tokens=30B, Evaluation Protocol=Static setting2025.1126.0824.45
ROCKETCR=0.4, Backbone=Llama3-8B2026.02298.1
SVD-LLMCR=0.2, Backbone=Llama3-8B2026.024111
CoSpaDiCR=0.3, Backbone=Llama3-8B2026.02459.2
SVD-LLMCR=0.3, Backbone=Llama3-8B2026.0215061
CoSpaDiCR=0.4, Backbone=Llama3-8B2026.02180120
SVD-LLMCR=0.4, Backbone=Llama3-8B2026.025501,300
CoSpaDiCR=0.5, Backbone=Llama3-8B2026.02740520
SVD-LLMCR=0.5, Backbone=Llama3-8B2026.021,8006,500