List your company
ResearchFormerly WizWand

Language Modeling on Wikitext-103 (PPL)

14.8Perplexity (PPL)GPT2 - M
10.56439.15767.7596.343Sep 17, 2024Dec 26, 2024Apr 6, 2025Jul 16, 2025Oct 24, 2025Feb 2, 2026May 14, 2026
Updated 5mo ago

Evaluation results

MethodLinks
GPT2 - MModel=GPT2 - M, Evaluation Protocol=Fine-Tuned, Bits per Token=7864322025.0514.8
ASTRAModel=GPT2 - M, #Groups=32, Evaluation Protocol=Fine-Tuned, Bits per Token=7680, Compression Ratio=102.42025.0516.84
ASTRAModel=GPT2 - M, #Groups=16, Evaluation Protocol=Fine-Tuned, Bits per Token=3840, Compression Ratio=204.82025.0518.03
GPT2 - SModel=GPT2 - S, Evaluation Protocol=Fine-Tuned, Bits per Token=2949122025.0518.96
ASTRAModel=GPT2 - S, #Groups=32, Evaluation Protocol=Fine-Tuned, Bits per Token=3840, Compression Ratio=76.82025.0520.95
ASTRAModel=GPT2 - M, #Groups=1, Evaluation Protocol=Fine-Tuned, Bits per Token=240, Compression Ratio=3276.82025.0521.97
ASTRAModel=GPT2 - S, #Groups=16, Evaluation Protocol=Fine-Tuned, Bits per Token=1920, Compression Ratio=153.62025.0523.22
ASTRAModel=GPT2 - S, #Groups=1, Evaluation Protocol=Fine-Tuned, Bits per Token=120, Compression Ratio=2457.62025.0525.98
BLIND-VALM-MModel=BLIND-VALM-M2024.0933.95
Manta-LMZero-shot=true, Protocol=ELBO-based proxy, Model parameters=101M2026.0535.35
MD4Zero-shot=true, Protocol=ELBO-based proxy2026.0535.9
RADDZero-shot=true, Protocol=ELBO-based proxy2026.0537.98
SEDDZero-shot=true, Protocol=ELBO-based proxy2026.0540.62
BLIND-VALM+Model=BLIND-VALM+2024.0940.95
AO-GPTZero-shot=true2026.0541.51
GPT-2Zero-shot=true2026.0541.6
GPT2 - MModel=GPT2 - M, Evaluation Protocol=Zero-Shot, Bits per Token=7864322025.0543.22
BLIND-VALMModel=BLIND-VALM2024.0943.46
VALMModel=VALM2024.0943.68
PLAIDZero-shot=true, Protocol=ELBO-based proxy2026.0550.86
GPT2 - SModel=GPT2 - S, Evaluation Protocol=Zero-Shot, Bits per Token=2949122025.0558.91
ASTRAModel=GPT2 - M, #Groups=32, Evaluation Protocol=Zero-Shot, Bits per Token=7680, Compression Ratio=102.42025.0562.29
D3PMZero-shot=true, Protocol=ELBO-based proxy2026.0575.16
ASTRAModel=GPT2 - M, #Groups=16, Evaluation Protocol=Zero-Shot, Bits per Token=3840, Compression Ratio=204.82025.0575.29
ASTRAModel=GPT2 - S, #Groups=32, Evaluation Protocol=Zero-Shot, Bits per Token=3840, Compression Ratio=76.82025.0576.24
ASTRAModel=GPT2 - S, #Groups=16, Evaluation Protocol=Zero-Shot, Bits per Token=1920, Compression Ratio=153.62025.0594.8
ASTRAModel=GPT2 - M, #Groups=1, Evaluation Protocol=Zero-Shot, Bits per Token=240, Compression Ratio=3276.82025.0596.99
ASTRAModel=GPT2 - S, #Groups=1, Evaluation Protocol=Zero-Shot, Bits per Token=120, Compression Ratio=2457.62025.05120.7