Sparse Autoencoding on Gemma-2-2B (layer 12)
0.9628KL ScoreC²R
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| C²Rlayer=12, k=100, independent runs=52026.06 | 0.9628 | 0.9309 | 0.0991 | 0.0656 | 1.0385 | |
| Batch TopKlayer=12, k=100, independent runs=52026.06 | 0.9621 | 0.9311 | 0.3325 | 0.1908 | 1.0962 | |
| Ortlayer=12, k=100, independent runs=52026.06 | 0.9615 | 0.9297 | 0.1038 | 0.0819 | 1.0577 | |
| Matryoshkalayer=12, k=100, independent runs=52026.06 | 0.9597 | 0.9311 | 0.187 | 0.036 | 1.0673 |