Language Modeling on LAMBADA
3.1PerplexityLlama3-8B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama3-8BBackbone=Llama3-8B, CR=None2026.02 | 3.1 | — | |
| GRINQH-8b-AnyPModel=Llama-3.1-8B-Instruct, Eff. Bits=3.892026.06 | 3.36 | — | |
| GRINQH-8b-AnyPModel=Llama-3.1-8B-Instruct, Eff. Bits=2.962026.06 | 3.37 | — | |
| GRINQH-8b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=4.012026.06 | 3.38 | — | |
| Llama2 7BCR=None2025.09 | 3.4 | — | |
| BF16-BaselineModel=Llama-3.1-8B-Instruct, Eff. Bits=162026.06 | 3.4 | — | |
| GRINQH-6b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=3.022026.06 | 3.4 | — | |
| NVFP4-G16Model=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 3.41 | — | |
| GRINQH-6b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=3.922026.06 | 3.41 | — | |
| SliM-LLMModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 3.46 | — | |
| Phi-4 14B (w/ LoopUS)Model=Phi-4 14B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 3.49 | — | |
| GRINQH-8b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=2.982026.06 | 3.49 | — | |
| QuaRot+GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 3.55 | — | |
| GRINQH-4b-GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=3.042026.06 | 3.56 | — | |
| GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 3.64 | — | |
| AnyPrecLLMModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 3.71 | — | |
| GRINQH-8b-AnyPModel=Llama-3.1-8B-Instruct, Eff. Bits=2.012026.06 | 3.72 | — | |
| GRINQH-4b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=3.022026.06 | 3.8 | — | |
| AutoRoundModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 3.89 | — | |
| RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 3.9 | — | |
| AWQModel=Llama-3.1-8B-Instruct, Eff. Bits=42026.06 | 3.91 | — | |
| Phi-4 14B (w/o LoopUS)Model=Phi-4 14B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 4.03 | — | |
| GRINQH-8b-AnyPModel=Qwen3-8B, Eff. Bits=2.222026.06 | 4.21 | — | |
| CoSpaDi (grouped)CR=0.22025.09 | 4.3 | — | |
| CoSpaDiBackbone=Llama3-8B, CR=0.22026.02 | 4.3 | — | |
| Qwen 8B (w/ LoopUS)Model=Qwen 8B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 4.32 | — | |
| GRINQH-8b-AnyPModel=Qwen3-8B, Eff. Bits=3.062026.06 | 4.35 | — | |
| GRINQH-8b-AnyPModel=Qwen3-8B, Eff. Bits=3.952026.06 | 4.44 | — | |
| CoSpaDi (per-layer)CR=0.22025.09 | 4.5 | — | |
| GRINQH-8b-RTNModel=Qwen3-8B, Eff. Bits=3.072026.06 | 4.51 | — | |
| GRINQH-6b-RTNModel=Qwen3-8B, Eff. Bits=3.062026.06 | 4.53 | — | |
| Qwen 8B (w/o LoopUS)Model=Qwen 8B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 4.58 | — | |
| GRINQH-8b-RTNModel=Qwen3-8B, Eff. Bits=3.972026.06 | 4.58 | — | |
| GRINQH-6b-RTNModel=Qwen3-8B, Eff. Bits=3.942026.06 | 4.58 | — | |
| BF16-BaselineModel=Qwen3-8B, Eff. Bits=162026.06 | 4.59 | — | |
| Qwen3 8B2026.02 | 4.6 | — | |
| Qwen3-8BBackbone=Qwen3-8B, CR=None2026.02 | 4.6 | — | |
| AnyPrecLLMModel=Qwen3-8B, Eff. Bits=42026.06 | 4.7 | — | |
| QuaRot+GPTQModel=Qwen3-8B, Eff. Bits=42026.06 | 4.81 | — | |
| AnyPrecLLMModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 4.83 | — | |
| SliM-LLMModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 4.85 | — | |
| CoSpaDiBackbone=Qwen3-8B, CR=0.22026.02 | 4.9 | — | |
| NVFP4-G16Model=Qwen3-8B, Eff. Bits=42026.06 | 5.07 | — | |
| AWQModel=Qwen3-8B, Eff. Bits=42026.06 | 5.09 | — | |
| GRINQH-4b-GPTQModel=Qwen3-8B, Eff. Bits=3.032026.06 | 5.1 | — | |
| GRINQH-8b-RTNModel=Qwen3-8B, Eff. Bits=2.252026.06 | 5.12 | — | |
| GPTQModel=Qwen3-8B, Eff. Bits=42026.06 | 5.17 | — | |
| COMPOTBackbone=Llama3-8B, CR=0.22026.02 | 5.2 | — | |
| SliM-LLMModel=Qwen3-8B, Eff. Bits=42026.06 | 5.2 | — | |
| AutoRoundModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 5.24 | — | |
| GRINQH-4b-RTNModel=Qwen3-8B, Eff. Bits=3.032026.06 | 5.3 | — | |
| Qwen 4B (w/ LoopUS)Model=Qwen 4B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 5.33 | — | |
| RTNModel=Qwen3-8B, Eff. Bits=42026.06 | 5.4 | — | |
| SliM-LLMModel=Qwen3-8B, Eff. Bits=32026.06 | 5.58 | — | |
| Llama3.2 1B2025.09 | 5.7 | — | |
| COMPOTBackbone=Qwen3-8B, CR=0.22026.02 | 5.7 | — | |
| QuaRot+GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 5.77 | — | |
| GPTQModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 5.86 | — | |
| GRINQH-8b-RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=2.112026.06 | 5.94 | — | |
| AutoRoundModel=Qwen3-8B, Eff. Bits=42026.06 | 5.94 | — | |
| COMPOTBackbone=Qwen3-8B, CR=0.32026.02 | 6.2 | — | |
| SVD-LLMCR=0.22025.09 | 6.3 | — | |
| CoSpaDi (grouped)CR=0.32025.09 | 6.3 | — | |
| CoSpaDiBackbone=Qwen3-8B, CR=0.32026.02 | 6.3 | — | |
| SVD-LLMBackbone=Qwen3-8B, CR=0.22026.02 | 6.4 | — | |
| AnyPrecLLMModel=Qwen3-8B, Eff. Bits=32026.06 | 6.5 | — | |
| CoSpaDi (per-layer)CR=0.32025.09 | 6.9 | — | |
| GPTQModel=Qwen3-8B, Eff. Bits=32026.06 | 6.96 | — | |
| Basis SharingCR=0.22025.09 | 7.2 | — | |
| Qwen 4B (w/o LoopUS)Model=Qwen 4B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 7.29 | — | |
| QuaRot+GPTQModel=Qwen3-8B, Eff. Bits=32026.06 | 7.3 | — | |
| AutoRoundModel=Qwen3-8B, Eff. Bits=32026.06 | 7.31 | — | |
| Qwen 1.7B (w/ LoopUS)Model=Qwen 1.7B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 7.43 | — | |
| RTNModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 7.43 | — | |
| Full KDDistillation Regime=Off-policy2026.02 | 7.8 | — | |
| RandomPos 20%Distillation Regime=Off-policy, Selection Budget=20%2026.02 | 8 | — | |
| SE-KDDistillation Regime=Off-policy2026.02 | 8 | — | |
| AWQModel=Llama-3.1-8B-Instruct, Eff. Bits=32026.06 | 8.04 | — | |
| COMPOTBackbone=Llama3-8B, CR=0.32026.02 | 8.5 | — | |
| TopSmp 20%Distillation Regime=Off-policy, Selection Budget=20%2026.02 | 8.6 | — | |
| SE-KD + TopSmp 20%Distillation Regime=Off-policy, Selection Budget=20%2026.02 | 8.6 | — | |
| SE-KD3XDistillation Regime=Off-policy2026.02 | 8.6 | — | |
| Pos RS-KD 20%Distillation Regime=Off-policy, Selection Budget=20%2026.02 | 8.9 | — | |
| Pos RS-KD* 20%Distillation Regime=Off-policy, Selection Budget=20%2026.02 | 9.2 | — | |
| CoSpaDiBackbone=Llama3-8B, CR=0.32026.02 | 9.2 | — | |
| SE-KDDistillation Regime=On-policy2026.02 | 9.5 | — | |
| COMPOTBackbone=Qwen3-8B, CR=0.42026.02 | 9.8 | — | |
| Full KDDistillation Regime=On-policy2026.02 | 10 | — | |
| RandomPos 20%Distillation Regime=On-policy, Selection Budget=20%2026.02 | 10 | — | |
| Pos RS-KD* 20%Distillation Regime=On-policy, Selection Budget=20%2026.02 | 10 | — | |
| TopSmp 20%Distillation Regime=On-policy, Selection Budget=20%2026.02 | 10.1 | — | |
| Hybrid Gated DeltaNet + M2RNN-5architecture=7B (1B active) MoE, setting=0-shot2026.03 | 10.29 | — | |
| Hybrid Gated DeltaNet + M2RNN-1architecture=7B (1B active) MoE, setting=0-shot2026.03 | 10.33 | — | |
| SE-KD + TopSmp 20%Distillation Regime=On-policy, Selection Budget=20%2026.02 | 10.4 | — | |
| Pos RS-KD 20%Distillation Regime=On-policy, Selection Budget=20%2026.02 | 10.5 | — | |
| Hybrid M2RNNarchitecture=7B (1B active) MoE, setting=0-shot2026.03 | 10.58 | — | |
| CleanTemperature=25°C, Noise Regime=low noise, Model=DeepSeek-MoE-16B2026.05 | 10.64 | — | |
| CleanTemperature=80°C, Noise Regime=high noise, Model=DeepSeek-MoE-16B2026.05 | 10.64 | — | |
| Hybrid Mamba-2 + M2RNN-1architecture=7B (1B active) MoE, setting=0-shot2026.03 | 10.84 | — | |
| SVD-LLMBackbone=Llama3-8B, CR=0.22026.02 | 11 | — |