Sparse Representation Learning on Llama 3.1 8B activations
127.8Achieved L0Top-K
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Top-KTarget L0=≈ 1282026.05 | 127.8 | 80.3 | 65.6 | 0.294 | 0.1 | 0.438 | 1 | |
| AEN-SAETarget L0=≈ 1282026.05 | 125.8 | 39.3 | 46.9 | 0.443 | 0.087 | 0.218 | 0.989 | |
| AEN-SAETarget L0=≈ 642026.05 | 68.8 | 45 | 40 | 0.503 | 0.093 | 0.182 | 0.862 | |
| Top-KTarget L0=≈ 642026.05 | 64 | 80.2 | 58 | 0.342 | 0.097 | 0.36 | 1 | |
| AEN-SAETarget L0=≈ 322026.05 | 34.5 | 55.4 | 19.9 | 0.671 | 0.097 | 0.143 | 0.585 | |
| Top-KTarget L0=≈ 322026.05 | 32 | 88 | 50.5 | 0.406 | 0.127 | 0.313 | 1 |