Correlation to Model Behavior Differences on MNLI
0.32Accuracy CorrelationSVCCA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SVCCAArchitecture=BERT2026.05 | 0.32 | 0.47 | 0 | |
| RSMDiffArchitecture=BERT2026.05 | 0.3 | -0.18 | -0.19 | |
| AngShapeArchitecture=BERT2026.05 | 0.28 | 0.26 | -0.02 | |
| OrthProcArchitecture=BERT2026.05 | 0.28 | 0.26 | -0.02 | |
| ProcDistArchitecture=BERT2026.05 | 0.28 | 0.07 | -0.38 | |
| Multi-Scale CKAArchitecture=BERT2026.05 | 0.27 | 0.33 | -0.08 | |
| AlignCosArchitecture=BERT2026.05 | 0.25 | 0.37 | -0.16 | |
| DistCorrArchitecture=ALBERT2026.05 | 0.25 | 0.32 | 0.57 | |
| Multi-Scale DistCorrArchitecture=ALBERT2026.05 | 0.25 | 0.27 | 0.56 | |
| RSAArchitecture=SmolLM22026.05 | 0.24 | -0.01 | 0 | |
| CKAArchitecture=SmolLM22026.05 | 0.22 | 0.08 | 0.12 | |
| MagDiffArchitecture=BERT2026.05 | 0.22 | 0.01 | -0.03 | |
| Multi-Scale CKAArchitecture=SmolLM22026.05 | 0.22 | 0.07 | 0.11 | |
| Multi-Scale DistCorrArchitecture=BERT2026.05 | 0.22 | 0.4 | -0.03 | |
| HardCorrArchitecture=ALBERT2026.05 | 0.21 | -0.03 | 0 | |
| AlignCosArchitecture=SmolLM22026.05 | 0.19 | 0.66 | 0.49 | |
| LinRegArchitecture=BERT2026.05 | 0.18 | 0.28 | -0.03 | |
| SoftCorrArchitecture=ALBERT2026.05 | 0.18 | -0.02 | 0.01 | |
| CKAArchitecture=BERT2026.05 | 0.18 | 0.3 | -0.01 | |
| DistCorrArchitecture=SmolLM22026.05 | 0.18 | 0.13 | 0.17 | |
| RSAArchitecture=ALBERT2026.05 | 0.18 | 0.23 | 0.47 | |
| SVCCAArchitecture=SmolLM22026.05 | 0.17 | 0.06 | 0.08 | |
| ProcDistArchitecture=SmolLM22026.05 | 0.17 | 0.6 | 0.43 | |
| CKAArchitecture=ALBERT2026.05 | 0.17 | 0.28 | 0.57 | |
| Multi-Scale DistCorrArchitecture=SmolLM22026.05 | 0.16 | 0.06 | 0.12 | |
| PermProcArchitecture=SmolLM22026.05 | 0.15 | 0.45 | 0.31 | |
| DistCorrArchitecture=BERT2026.05 | 0.15 | 0.39 | 0.12 | |
| AngShapeArchitecture=SmolLM22026.05 | 0.14 | 0.39 | 0.23 | |
| OrthProcArchitecture=SmolLM22026.05 | 0.14 | 0.39 | 0.23 | |
| UnifDiffArchitecture=BERT2026.05 | 0.14 | -0.02 | -0.14 | |
| Multi-Scale CKAArchitecture=ALBERT2026.05 | 0.14 | 0.26 | 0.53 | |
| AD-RBF-CKAArchitecture=SmolLM22026.05 | 0.13 | 0.24 | 0.24 | |
| AngShapeArchitecture=ALBERT2026.05 | 0.12 | -0.08 | -0.01 | |
| OrthProcArchitecture=ALBERT2026.05 | 0.12 | -0.08 | -0.01 | |
| Multi-Scale RBF-CKAArchitecture=ALBERT2026.05 | 0.12 | 0.04 | 0.17 | |
| SoftCorrArchitecture=BERT2026.05 | 0.11 | -0.23 | -0.42 | |
| Multi-Scale RBF-CKAArchitecture=SmolLM22026.05 | 0.1 | 0.26 | 0.19 | |
| PermProcArchitecture=BERT2026.05 | 0.09 | -0.06 | -0.3 | |
| RBF-CKAArchitecture=SmolLM22026.05 | 0.09 | 0.36 | 0.29 | |
| RTDArchitecture=BERT2026.05 | 0.09 | 0.04 | 0.1 | |
| SoftCorrArchitecture=SmolLM22026.05 | 0.07 | 0.61 | 0.37 | |
| RBF-CKAArchitecture=BERT2026.05 | 0.07 | 0.27 | 0.54 | |
| RSMDiffArchitecture=SmolLM22026.05 | 0.06 | 0.35 | 0.24 | |
| LinRegArchitecture=ALBERT2026.05 | 0.05 | 0.01 | -0.13 | |
| PWCCAArchitecture=SmolLM22026.05 | 0.04 | 0.53 | 0.32 | |
| HardCorrArchitecture=BERT2026.05 | 0.04 | -0.27 | -0.43 | |
| RBF-CKAArchitecture=ALBERT2026.05 | 0.03 | -0.01 | 0.13 | |
| EOSArchitecture=BERT2026.05 | 0.03 | 0.36 | 0.01 | |
| 2nd-CosArchitecture=SmolLM22026.05 | 0.03 | -0.07 | 0.01 | |
| AD-CKAArchitecture=SmolLM22026.05 | 0.02 | -0.31 | -0.21 | |
| Multi-Scale RBF-CKAArchitecture=BERT2026.05 | 0.02 | 0.2 | 0.43 | |
| PWCCAArchitecture=BERT2026.05 | 0.01 | 0.22 | -0.37 | |
| HardCorrArchitecture=SmolLM22026.05 | 0.01 | 0.5 | 0.3 | |
| GULPArchitecture=SmolLM22026.05 | 0.01 | 0.25 | 0.13 | |
| ConcDiffArchitecture=SmolLM22026.05 | 0.01 | 0.46 | 0.3 | |
| AD-DistCorrArchitecture=SmolLM22026.05 | 0.01 | -0.34 | -0.19 | |
| SVCCAArchitecture=ALBERT2026.05 | 0 | 0.12 | 0.33 | |
| AlignCosArchitecture=ALBERT2026.05 | 0 | 0.09 | -0.03 | |
| RSAArchitecture=BERT2026.05 | 0 | 0.27 | 0.19 | |
| ConcDiffArchitecture=BERT2026.05 | 0 | 0.02 | -0.31 | |
| LinRegArchitecture=SmolLM22026.05 | -0.01 | 0.62 | 0.33 | |
| GULPArchitecture=BERT2026.05 | -0.01 | 0.35 | 0.02 | |
| PermProcArchitecture=ALBERT2026.05 | -0.02 | 0.04 | -0.04 | |
| RTDArchitecture=SmolLM22026.05 | -0.02 | 0.33 | 0.12 | |
| PWCCAArchitecture=ALBERT2026.05 | -0.03 | -0.17 | -0.36 | |
| JaccardArchitecture=SmolLM22026.05 | -0.03 | 0.09 | -0.03 | |
| ProcDistArchitecture=ALBERT2026.05 | -0.04 | 0 | -0.07 | |
| RankSimArchitecture=SmolLM22026.05 | -0.05 | 0.08 | -0.01 | |
| MagDiffArchitecture=ALBERT2026.05 | -0.06 | 0.01 | 0.08 | |
| ConcDiffArchitecture=ALBERT2026.05 | -0.07 | -0.07 | 0.07 | |
| IMDArchitecture=ALBERT2026.05 | -0.08 | -0.29 | -0.16 | |
| RankSimArchitecture=BERT2026.05 | -0.09 | 0.08 | 0.05 | |
| AD-CKAArchitecture=BERT2026.05 | -0.09 | -0.02 | -0.06 | |
| EOSArchitecture=ALBERT2026.05 | -0.1 | -0.1 | -0.16 | |
| EOSArchitecture=SmolLM22026.05 | -0.11 | -0.3 | -0.23 | |
| AD-DistCorrArchitecture=BERT2026.05 | -0.11 | 0.03 | -0.03 | |
| GULPArchitecture=ALBERT2026.05 | -0.12 | -0.1 | -0.16 | |
| UnifDiffArchitecture=SmolLM22026.05 | -0.12 | -0.05 | 0.03 | |
| IMDArchitecture=SmolLM22026.05 | -0.13 | 0.12 | 0.09 | |
| AD-RBF-CKAArchitecture=BERT2026.05 | -0.14 | 0.05 | -0.05 | |
| RSMDiffArchitecture=ALBERT2026.05 | -0.15 | -0.02 | 0.12 | |
| AD-RBF-CKAArchitecture=ALBERT2026.05 | -0.15 | 0.13 | 0.3 | |
| UnifDiffArchitecture=ALBERT2026.05 | -0.16 | -0.3 | -0.24 | |
| MagDiffArchitecture=SmolLM22026.05 | -0.2 | -0.39 | -0.46 | |
| JaccardArchitecture=BERT2026.05 | -0.21 | 0.13 | 0.17 | |
| AD-DistCorrArchitecture=ALBERT2026.05 | -0.23 | 0.19 | 0.35 | |
| 2nd-CosArchitecture=ALBERT2026.05 | -0.25 | 0.04 | 0.12 | |
| JaccardArchitecture=ALBERT2026.05 | -0.25 | 0.02 | -0.05 | |
| AD-CKAArchitecture=ALBERT2026.05 | -0.25 | 0.24 | 0.34 | |
| 2nd-CosArchitecture=BERT2026.05 | -0.26 | 0.16 | 0.55 | |
| IMDArchitecture=BERT2026.05 | -0.26 | -0.39 | -0.06 | |
| RankSimArchitecture=ALBERT2026.05 | -0.27 | -0.06 | -0.13 | |
| RTDArchitecture=ALBERT2026.05 | -0.27 | -0.34 | -0.18 |