Sparse Autoencoder Evaluation on Gemma-2-2B activations
320L0 CountTopK
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TopKk=3202026.05 | 320 | 92.7 | 4.8 | 1.115 | 13.1 | 17.4 | |
| SoftSAEk=3202026.05 | 302.355 | 92 | 2.9 | 1.308 | 17.8 | 25 | |
| Batch TopKk=3202026.05 | 302.343 | 92.8 | 7.2 | 1.308 | 14.5 | 6.2 | |
| Matryoshkak=3202026.05 | 297.323 | 91.4 | 11 | 1.231 | 21 | 17.8 | |
| TopKk=1602026.05 | 160 | 90.4 | 6 | 1 | 11.4 | 6.5 | |
| SoftSAEk=1602026.05 | 152.657 | 89.1 | 3.8 | 1.115 | 13 | 6 | |
| Batch TopKk=1602026.05 | 150.634 | 90.4 | 6.4 | 1.038 | 9.8 | 4.4 | |
| Matryoshkak=1602026.05 | 149.306 | 89.2 | 0.4 | 1.308 | 21 | 9.5 | |
| TopKk=802026.05 | 80 | 88.2 | 15.9 | 1.231 | 10.1 | 2.6 | |
| Batch TopKk=802026.05 | 75.649 | 88.2 | 11.6 | 1.038 | 11.1 | 2.3 | |
| Matryoshkak=802026.05 | 74.956 | 86.9 | 1.5 | 1.154 | 20.3 | 6.5 | |
| SoftSAEk=802026.05 | 74.38 | 87.4 | 9.3 | 1 | 10 | 1.7 | |
| TopKk=402026.05 | 40 | 85.7 | 27.5 | 2.154 | 10.8 | 2.2 | |
| Batch TopKk=402026.05 | 38.326 | 85.7 | 26.7 | 1.808 | 10.9 | 2.3 | |
| Matryoshkak=402026.05 | 37.247 | 84.5 | 5.6 | 1.038 | 19.5 | 4.4 | |
| SoftSAEk=402026.05 | 36.668 | 84.6 | 22.6 | 1.346 | 13.9 | 2.3 | |
| TopKk=202026.05 | 20 | 82.8 | 25.5 | 3.038 | 10 | 1.4 | |
| Matryoshkak=202026.05 | 19.531 | 81.8 | 10.5 | 1.115 | 17.8 | 2.2 | |
| Batch TopKk=202026.05 | 19.452 | 83.1 | 31.7 | 2.885 | 10.3 | 1.3 | |
| SoftSAEk=202026.05 | 17.977 | 82.1 | 29.9 | 2.385 | 10.7 | 1.5 |