Activation Reconstruction on Pythia model activations
0.8074Pearson Correlation CoefficientLayerNorm
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LayerNormModel=pythia-2.8b2026.01 | 0.8074 | 3.5708 | |
| GroupNormModel=pythia-2.8b2026.01 | 0.8038 | 7.1652 | |
| LayerNormModel=pythia-1.4b2026.01 | 0.7958 | 5.2756 | |
| GroupNormModel=pythia-1.4b2026.01 | 0.7914 | 6.3021 | |
| BatchNormModel=pythia-2.8b2026.01 | 0.763 | 2.4657 | |
| PannealModel=pythia-2.8b2026.01 | 0.7613 | 6.3563 | |
| BatchNormModel=pythia-1.4b2026.01 | 0.7543 | 1.6418 | |
| PannealModel=pythia-1.4b2026.01 | 0.7413 | 5.7218 | |
| LayerNormModel=pythia-70m2026.01 | 0.7325 | 7.6272 | |
| GroupNormModel=pythia-70m2026.01 | 0.7285 | 9.0071 | |
| TopKModel=pythia-2.8b2026.01 | 0.7074 | 22.5243 | |
| Batch-TopKModel=pythia-2.8b2026.01 | 0.7037 | 23.769 | |
| TopKModel=pythia-1.4b2026.01 | 0.6969 | 22.7108 | |
| Batch-TopKModel=pythia-1.4b2026.01 | 0.6856 | 24.385 | |
| BatchNormModel=pythia-70m2026.01 | 0.65 | 8.3305 | |
| PannealModel=pythia-70m2026.01 | 0.6474 | 7.7097 | |
| TopKModel=pythia-70m2026.01 | 0.5851 | 22.9035 | |
| Batch-TopKModel=pythia-70m2026.01 | 0.5687 | 24.6165 |