Activation Reconstruction on Wikipedia activations Gemma-2-2B
0.592MSESAE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SAELayer=6, Number of Features (K)=65K2026.02 | 0.592 | — | |
| MFALayer=6, Number of Features (K)=32K2026.02 | 0.8 | — | |
| MFALayer=6, Number of Features (K)=8K2026.02 | 0.896 | — | |
| MFALayer=6, Number of Features (K)=1K2026.02 | 1.098 | — | |
| K-MeansLayer=6, Number of Features (K)=32K2026.02 | 1.104 | 1.4 | |
| K-MeansLayer=6, Number of Features (K)=8K2026.02 | 1.265 | 1.4 | |
| K-MeansLayer=6, Number of Features (K)=1K2026.02 | 1.53 | 1.4 | |
| MFALayer=18, Number of Features (K)=32K2026.02 | 4.477 | — | |
| SAELayer=18, Number of Features (K)=65K2026.02 | 4.595 | — | |
| MFALayer=18, Number of Features (K)=8K2026.02 | 4.678 | — | |
| MFALayer=18, Number of Features (K)=1K2026.02 | 5.43 | — | |
| K-MeansLayer=18, Number of Features (K)=32K2026.02 | 5.966 | 1.3 | |
| K-MeansLayer=18, Number of Features (K)=8K2026.02 | 6.622 | 1.4 | |
| K-MeansLayer=18, Number of Features (K)=1K2026.02 | 7.629 | 1.4 |