Sparse Autoencoder Reconstruction on WikiText-103 Llama-3.1-8B activations v1 (test)
0.057MSEStandard SAE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Standard SAEModel=Llama-3.1-8B2026.06 | 0.057 | 0.931 | 3.55 | 90.6 | |
| VASAEModel=Llama-3.1-8B2026.06 | 0.058 | 0.931 | 3.55 | 90.6 | |
| Hard-tied decoderModel=Llama-3.1-8B2026.06 | 0.672 | 0.021 | 11.46 | 3.2 |