List your company
ResearchFormerly WizWand

Language Modeling on Language Modeling Dataset

8.31PPLO-FANG
4.372430.951257.5384.1088Dec 8, 2025Jan 11, 2026Feb 14, 2026Mar 20, 2026Apr 23, 2026May 27, 2026Jul 1, 2026
Updated 4mo ago

Evaluation results

MethodLinks
O-FANGModel=LLaMA-3.1-8B, Sparsity=20%2025.128.31—
FP16Bits=16.02026.078.74—
TASA b4.0Bits=4.02026.079.02—
SpQR†Bits=4.12 (W3)2026.079.06—
GPTQBits=4.02026.079.1—
AWQBits=4.02026.079.13—
OBCModel=LLaMA-3.1-8B, Sparsity=20%2025.129.17—
HQQBits=4.02026.079.23—
RTNBits=4.02026.079.75—
TASA b3.0Bits=3.02026.079.96—
OWQBits=3.02026.0710.16—
GPTQBits=3.02026.0710.69—
HQQBits=3.02026.0711.65—
O-FANGModel=Qwen2.5-7B, Sparsity=40%2025.1211.72—
DensePrune%=0%, Zero-shot=true, Backbone=LLaMA-2-7B2025.1212.18—
OBCModel=Qwen2.5-7B, Sparsity=40%2025.1214.04—
PPPrune%=20%, Zero-shot=true, Backbone=LLaMA-2-7B2025.1215.31—
FLAPPrune%=20%, Zero-shot=true, Backbone=LLaMA-2-7B2025.1216.32—
SlimGPT w/oPrune%=20%, Zero-shot=true, Backbone=LLaMA-2-7B2025.1216.36—
Token FilteringPrune%=20%, Zero-shot=true, Backbone=LLaMA-2-7B2025.1216.65—
GPT-2 XLParams=1.5B2026.0517.48—
CCT-306M (RC2)Params=306M2026.0521.27—
GPT-2 LargeParams=774M2026.0522.05—
Transformer-XLParams=257M2026.0524—
GPT-2 MediumParams=355M2026.0526.37—
GPT-2 SmallParams=124M2026.0537.5—
PPPrune%=50%, Zero-shot=true, Backbone=LLaMA-2-7B2025.1239.4—
SlimGPT w/oPrune%=50%, Zero-shot=true, Backbone=LLaMA-2-7B2025.1240.83—
FLAPPrune%=50%, Zero-shot=true, Backbone=LLaMA-2-7B2025.1243.11—
Token FilteringPrune%=50%, Zero-shot=true, Backbone=LLaMA-2-7B2025.1254.59—
RTNBits=3.02026.07106.75—
Adam-DecayModel size=7 B, Inner steps=240, Staleness (τ)=0, Number of seeds (n)=22026.05—11.28
Adam-DecayModel size=7 B, Inner steps=240, Staleness (τ)=8, Number of seeds (n)=22026.05—11.3
Adam-DecayModel size=7 B, Inner steps=240, Staleness (τ)=16, Number of seeds (n)=22026.05—11.69
CGADModel size=25 M, Staleness (τ)=0, Number of seeds (n)=3–82026.05—6.79
CGADModel size=25 M, Staleness (τ)=8, Number of seeds (n)=3–82026.05—8.46
CGADModel size=25 M, Staleness (τ)=16, Number of seeds (n)=3–82026.05—8.96
CGADModel size=1 B, Staleness (τ)=0, Number of seeds (n)=42026.05—7.48
CGADModel size=1 B, Staleness (τ)=8, Number of seeds (n)=32026.05—9.94
CGADModel size=1 B, Staleness (τ)=16, Number of seeds (n)=12026.05—7.44
CGADModel size=7 B, Inner steps=240, Staleness (τ)=0, Number of seeds (n)=42026.05—10.48
CGADModel size=7 B, Inner steps=240, Staleness (τ)=8, Number of seeds (n)=32026.05—10.9
CGADModel size=7 B, Inner steps=240, Staleness (τ)=16, Number of seeds (n)=22026.05—11.7
DelayedNesterovModel size=7 B, Inner steps=240, Staleness (τ)=0, Number of seeds (n)=22026.05—11.1
DelayedNesterovModel size=7 B, Inner steps=240, Staleness (τ)=8, Number of seeds (n)=22026.05—10.39
DelayedNesterovModel size=7 B, Inner steps=240, Staleness (τ)=16, Number of seeds (n)=22026.05—11.69
NesterovModel size=25 M, Staleness (τ)=0, Number of seeds (n)=3–52026.05—8.04
NesterovModel size=25 M, Staleness (τ)=8, Number of seeds (n)=3–52026.05—170.9
NesterovModel size=25 M, Staleness (τ)=16, Number of seeds (n)=3–52026.05—487.6
NesterovModel size=1 B, Staleness (τ)=0, Number of seeds (n)=52026.05—9.7
NesterovModel size=1 B, Staleness (τ)=8, Number of seeds (n)=32026.05—241.2
NesterovModel size=1 B, Staleness (τ)=16, Number of seeds (n)=22026.05—319
NesterovModel size=7 B, Inner steps=240, Staleness (τ)=0, Number of seeds (n)=42026.05—46.69
NesterovModel size=7 B, Inner steps=240, Staleness (τ)=8, Number of seeds (n)=12026.05—65.31
SDMModel size=7 B, Inner steps=240, Staleness (τ)=0, Number of seeds (n)=22026.05—11.3
SDMModel size=7 B, Inner steps=240, Staleness (τ)=8, Number of seeds (n)=22026.05—11.44
SDMModel size=7 B, Inner steps=240, Staleness (τ)=16, Number of seeds (n)=22026.05—11.69