# Sparse Representation Learning on Llama 3.1 8B activations

Source: [Sparse Representation Learning on Llama 3.1 8B activations](<https://www.sota2.com/research/sota/sparse-representation-learning-on-llama-3-1-8b-activations>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-08T07:37:06.566Z

Evaluation of the internal representation quality and sparsity characteristics (L0, Coherence, Explained Variance) derived from the activations of the Llama 3.1 8B model.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Sparse representation learning |
| Dataset (including any stated split) | Llama 3.1 8B activations |
| Metric | Achieved L0 |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Achieved L0](<https://www.sota2.com/research/sota/sparse-representation-learning-on-llama-3-1-8b-activations.md?metric=Achieved%20L0>): Higher is better; unit: SCALAR.

- [Dead Features (%)](<https://www.sota2.com/research/sota/sparse-representation-learning-on-llama-3-1-8b-activations.md?metric=Dead%20Features%20(%25)>): Lower is better; unit: PERCENT.

- [Explained Variance](<https://www.sota2.com/research/sota/sparse-representation-learning-on-llama-3-1-8b-activations.md?metric=Explained%20Variance>): Higher is better; unit: PERCENT.

- [Mean Squared Error](<https://www.sota2.com/research/sota/sparse-representation-learning-on-llama-3-1-8b-activations.md?metric=Mean%20Squared%20Error>): Lower is better; unit: SCALAR.

- [Coherence (p50)](<https://www.sota2.com/research/sota/sparse-representation-learning-on-llama-3-1-8b-activations.md?metric=Coherence%20(p50)>): Higher is better; unit: SCALAR.

- [Coherence (p90)](<https://www.sota2.com/research/sota/sparse-representation-learning-on-llama-3-1-8b-activations.md?metric=Coherence%20(p90)>): Higher is better; unit: SCALAR.

- [Coherence (Max)](<https://www.sota2.com/research/sota/sparse-representation-learning-on-llama-3-1-8b-activations.md?metric=Coherence%20(Max)>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Achieved L0 | Result date | Report paper |
| --- | --- | --- | --- |
| Top-K | 127.8 | Not recorded | 69fd6de35497b3b670c89265 |
| AEN-SAE | 125.8 | Not recorded | 69fd6de35497b3b670c89265 |
| AEN-SAE | 68.8 | Not recorded | 69fd6de35497b3b670c89265 |
| Top-K | 64 | Not recorded | 69fd6de35497b3b670c89265 |
| AEN-SAE | 34.5 | Not recorded | 69fd6de35497b3b670c89265 |
| Top-K | 32 | Not recorded | 69fd6de35497b3b670c89265 |

### Top-K

Achieved L0: 127.8. Result ID: 69fd924247f34ce924e03e75.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6de35497b3b670c89265.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738d1e209131cea5d2c9.md>)

Recorded evaluation configuration: {"Target L0":"≈ 128"}. This record does not assert independent reproduction.

### AEN-SAE

Achieved L0: 125.8. Result ID: 69fd924247f34ce924e03e76.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6de35497b3b670c89265.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fd6de35497b3b670c89265.md>)

Recorded evaluation configuration: {"Target L0":"≈ 128"}. This record does not assert independent reproduction.

### AEN-SAE

Achieved L0: 68.8. Result ID: 69fd924247f34ce924e03e74.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6de35497b3b670c89265.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fd6de35497b3b670c89265.md>)

Recorded evaluation configuration: {"Target L0":"≈ 64"}. This record does not assert independent reproduction.

### Top-K

Achieved L0: 64. Result ID: 69fd924247f34ce924e03e73.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6de35497b3b670c89265.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738d1e209131cea5d2c9.md>)

Recorded evaluation configuration: {"Target L0":"≈ 64"}. This record does not assert independent reproduction.

### AEN-SAE

Achieved L0: 34.5. Result ID: 69fd924247f34ce924e03e72.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6de35497b3b670c89265.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69fd6de35497b3b670c89265.md>)

Recorded evaluation configuration: {"Target L0":"≈ 32"}. This record does not assert independent reproduction.

### Top-K

Achieved L0: 32. Result ID: 69fd924247f34ce924e03e71.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6de35497b3b670c89265.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738d1e209131cea5d2c9.md>)

Recorded evaluation configuration: {"Target L0":"≈ 32"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.