List your company

ResearchFormerly WizWand

Sparse Autoencoding

Benchmarks

DatasetSOTA methodMetricTrendResultsLast updated

Pythia-70M layer 3

0.342Relative Error

13

Jul 3, 2026

Pythia-70M residual-stream activations 5,000-token (test)

0.341Relative Error

12

Jul 3, 2026

Pythia-160M residual-stream activations 5,000-token (test)

0.332Relative Error

8

Jul 3, 2026

gemma2-2b-layer-13 activations

100.7L0

6

Feb 26, 2026

Gemma-2-2B activations (layer 20)

0.973KL Score

5

Jun 30, 2026

Llama-3.2-1B Layer 12 activations (held-out)

0.499Relative Error

4

Jul 3, 2026

Llama-3.2-1B Layer 6 activations (held-out)

56.8Relative Error

4

Jul 3, 2026

Qwen2.5-3B Layer 24 activations (held-out)

0.591Relative Error

4

Jul 3, 2026

Qwen2.5-3B Layer 12 activations (held-out)

48.9Relative Error

4

Jul 3, 2026

Pythia-160M Layer 8 activations (held-out)

0.332Relative Reconstruction Error

4

Jul 3, 2026

Pythia-70M Layer 3 activations (held-out)

0.342Relative Error

4

Jul 3, 2026

Gemma-2-2B (layer 12)

0.963KL Score

4

Jun 30, 2026

DINOv2 ImageNet-1k Base (patch tokens)

0.191MSE

4

Jun 1, 2026

Pythia activations (The Pile) 160M (layer 12)

0.148MSE

4

Jun 1, 2026

Pythia 70M (layer-3 residuals)

33.9Achieved L0

4

May 8, 2026

Mistral-7B activations

95.3KL Score

2

Jun 5, 2026

GPT-2 activations

98KL Score

2

Jun 5, 2026

qwen3-4b layer-18 activations

50L0 Norm

2

Feb 26, 2026

gemma2-2b layer-20 activations

50L0 Norm

2

Feb 26, 2026

gemma2-2B-layer-6 activations

50L0 Norm

2

Feb 26, 2026

SlimPajama, Proprietary, and FineWeb-Edu

—

—Primary metric

0

Jun 26, 2026