Sparse Autoencoder Reconstruction on WikiText-103 GPT-2-small activations v1 (test)
2.04MSEStandard SAE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Standard SAEModel=GPT-2-small2026.06 | 2.04 | 0.965 | 4.18 | 0.975 | |
| VASAEModel=GPT-2-small2026.06 | 2.04 | 0.965 | 4.17 | 0.975 | |
| Hard-tied decoderModel=GPT-2-small2026.06 | 110.39 | 0.434 | 14.64 | 0.606 |