List your company

Language Modeling on LM-Harness (Standard Suite Metrics)

32.85ARC-C AccuracyDeltaNet
23.708426.081728.45530.8283Nov 26, 2025

Evaluation results

MethodLinks
DeltaNetModel Scale=2.8B, Training Tokens=100B2025.1132.8558.1642.518161.1373.7843.961.7211.846.0851.29
Gated DeltaNetModel Scale=2.8B, Training Tokens=100B2025.1132.5960.0262.758262.874.3280.662.358.2644.2857
Gated KalmaNetModel Scale=2.8B, Training Tokens=100B2025.1132.5159.8961.688563.8474.8183.264.1712.8950.9558.89
TransformerModel Scale=2.8B, Training Tokens=100B2025.1132.2556.164.288060.9673.5679.561.7258.5372.2863.92
Mamba2Model Scale=2.8B, Training Tokens=100B2025.1132.2459.6458.728262.2373.7879.862.197.7141.1355.94
Mamba2Model Scale=1B, Training Tokens=20B2025.1128.0746.6360.217044.5767.5765.554.31.4515.7545.4
Gated Linear AttentionModel Scale=2.8B, Training Tokens=100B2025.1127.8250.852.577848.8370.1369.654.542.8120.4347.55
DeltaNetModel Scale=1B, Training Tokens=20B2025.1127.5646.2559.977143.1867.7465.955.413.0920.6146.07
Gated DeltaNetModel Scale=1B, Training Tokens=20B2025.1127.0547.9859.547444.2767.3666.253.832.1817.8246.02
TransformerModel Scale=1B, Training Tokens=20B2025.1126.6246.4259.947744.0167.1468.354.068.3545.1849.7
Gated Linear AttentionModel Scale=1B, Training Tokens=20B2025.1125.944258.847036.3463.658.251.851.4510.5341.88
Gated KalmaNetModel Scale=1B, Training Tokens=20B2025.1125.4346.5560.737444.5968.8867.652.416.1721.8746.82
Gated DeltaNetModel Scale=440M, Training Tokens=8B2025.1125.1741.9658.237236.9664.6963.651.71.9111.8842.81
DeltaNetModel Scale=440M, Training Tokens=8B2025.1125.0941.9261.136537.264.476449.492.8114.3142.54
Gated KalmaNetModel Scale=440M, Training Tokens=8B2025.1124.5743.2256.947137.2264.4762.850.831.4514.0442.65
TransformerModel Scale=440M, Training Tokens=8B2025.1124.442.2659.887036.1964.1561.551.75.1735.6445.09
Gated Linear AttentionModel Scale=440M, Training Tokens=8B2025.1124.0640.2856.577132.762.2457.850.6719.1840.55