Downstream Behavior Evaluation on Pythia-160m Layer 8 Residual-Stream Activations
9.7Delta CERSAE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| RSAEActivation Function Family=TopK2026.06 | 9.7 | 98.31 | |
| RSAEActivation Function Family=JumpReLU2026.06 | 11.8 | 98.1 | |
| TopK SAEActivation Function Family=TopK2026.06 | 23.2 | 95.94 | |
| ReLU SAEActivation Function Family=ReLU2026.06 | 25.9 | 95.71 | |
| RSAEActivation Function Family=ReLU2026.06 | 27.9 | 95.41 | |
| JumpReLU SAEActivation Function Family=JumpReLU2026.06 | 68.2 | 88.86 |