Activation Reconstruction on Pythia 1.4b
0.22MSESAE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SAEVariant=SAE2026.05 | 0.22 | 0.9784 | 0.9289 | 0.8 | — | |
| fmxcoderVariant=fmxcoder (p = 0), masking_probability=02026.05 | 0.22 | 0.9781 | 0.9275 | 10.8 | — | |
| fmxcoderVariant=fmxcoder (p = 0.05), masking_probability=0.052026.05 | 0.23 | 0.9777 | 0.9263 | 5.5 | — | |
| CrosscoderVariant=Crosscoder2026.05 | 0.31 | 0.9691 | 0.8976 | 0.2 | — | |
| Basis-Sharing-coreBackbone=Pythia 1.4B, Compression Ratio (d/D)=0.252026.05 | — | — | — | — | 7.95 | |
| Pure-bundle GBD (λ=0)Backbone=Pythia 1.4B, Compression Ratio (d/D)=0.252026.05 | — | — | — | — | 5.76 | |
| SVD-LLM V1Backbone=Pythia 1.4B, Compression Ratio (d/D)=0.252026.05 | — | — | — | — | 1.93 | |
| SVD-LLM V2Backbone=Pythia 1.4B, Compression Ratio (d/D)=0.252026.05 | — | — | — | — | 1.79 |