Sparse Autoencoder Disentanglement and Interpretability on Llama-3-8B (layer 20)
0.986KL ScoreBatch TopK
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Batch TopKlayer=20, k=1002026.06 | 0.986 | 0.9405 | 0.3334 | 0.147 | 1.1923 | |
| Ortlayer=20, k=1002026.06 | 0.9859 | 0.9265 | 0.12 | 0.0979 | 1.0385 | |
| C^2Rlayer=20, k=1002026.06 | 0.9858 | 0.9443 | 0.0816 | 0.041 | 1.0385 | |
| Matryoshkalayer=20, k=1002026.06 | 0.9853 | 0.9552 | 0.1803 | 0.0349 | 1.1154 |