Observability Analysis on WikiText-103 (val)
0.388Partial Correlation (rho)Gemma 3 1B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemma 3 1BFamily=Gemma, Params=1B, Peak layer=L1 (4%), Token budget=150 ex/dim, n_seeds=72026.04 | 0.388 | 0.004 | 0.307 | 0.98 | 0.213 | |
| Mistral 7BFamily=Mistral, Params=7B, Peak layer=L22 (69%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.313 | 0.001 | 0.156 | 0.995 | 0.014 | |
| Phi-3 MiniFamily=Phi, Params=3.8B, Peak layer=L17 (53%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.3 | 0.002 | 0.144 | 0.958 | 0.008 | |
| GPT-2 XLFamily=GPT-2, Params=1.5B, Peak layer=L34 (71%), Token budget=350 ex/dim, n_seeds=32026.04 | 0.29 | 0.004 | 0.174 | 0.952 | — | |
| LLAMA 3.2 1BFamily=Llama, Params=1.2B, Peak layer=L13 (81%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.286 | 0.006 | 0.12 | 0.995 | 0.012 | |
| QWEN 2.5 1.5BFamily=Qwen, Params=1.5B, Peak layer=L18 (64%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.275 | 0.032 | 0.127 | 0.953 | 0.051 | |
| QWEN 2.5 3BFamily=Qwen, Params=3B, Peak layer=L25 (69%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.263 | 0.021 | 0.144 | 0.925 | 0.023 | |
| QWEN 2.5 7BFamily=Qwen, Params=7B, Peak layer=L17 (61%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.255 | 0.019 | 0.137 | 0.964 | — | |
| QWEN 2.5 0.5BFamily=Qwen, Params=0.5B, Peak layer=L19 (79%), Token budget=600 ex/dim, n_seeds=72026.04 | 0.215 | 0.02 | 0.059 | 0.959 | 0.055 | |
| QWEN 2.5 14BFamily=Qwen, Params=14B, Peak layer=L30 (62%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.214 | 0.032 | 0.096 | 0.851 | 0.009 | |
| Gemma 3 4BFamily=Gemma, Params=4.3B, Peak layer=L6 (18%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.191 | 0.007 | 0.059 | 0.802 | 0.01 | |
| LLAMA 3.1 8BFamily=Llama, Params=8B, Peak layer=L1 (3%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.093 | 0.012 | 0.002 | 0.994 | 0.005 | |
| LLAMA 3.2 3BFamily=Llama, Params=3B, Peak layer=L0 (0%), Token budget=350 ex/dim, n_seeds=72026.04 | 0.091 | 0.006 | 0.031 | 0.998 | 0.002 |