Interpretability and Faithfulness Evaluation on DINOv2 ViT-B/14 tokens
10LLM RankJumpReLU
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| JumpReLUBackbone=DINOv2 ViT-B/14, Expansion factor=82026.05 | 10 | 22.59 | 84.27 | 99.97 | |
| TranscoderBackbone=DINOv2 ViT-B/14, Expansion factor=82026.05 | 9 | 50.36 | 79.63 | 83.1 | |
| Vanilla SAEBackbone=DINOv2 ViT-B/14, Expansion factor=82026.05 | 8 | 19.32 | 77.63 | 47.76 | |
| Backbone onlyBackbone=DINOv2 ViT-B/142026.05 | 7 | 36.2 | 84.3 | 100 | |
| TopK-SAEBackbone=DINOv2 ViT-B/14, Expansion factor=82026.05 | 6 | 47.63 | 82.19 | 88.58 | |
| DINOv2 ViT-B/14 layer 10 (backbone only)Backbone=DINOv2 ViT-B/14, Layer=102026.05 | 6 | 37.02 | 84.3 | 100 | |
| DINOv2 ViT-B/14 layer 11 (backbone only)Backbone=DINOv2 ViT-B/14, Layer=112026.05 | 6 | 30.7 | 84.3 | 100 | |
| Matryoshka SAEBackbone=DINOv2 ViT-B/14, Expansion factor=82026.05 | 5 | 52.14 | 82.13 | 88.71 | |
| Archetypal SAEBackbone=DINOv2 ViT-B/14, Layer=102026.05 | 5 | 46.66 | 81.33 | 68.99 | |
| Matching PursuitBackbone=DINOv2 ViT-B/14, Layer=112026.05 | 5 | 53.91 | 83.89 | 86.46 | |
| BatchTopK-SAEBackbone=DINOv2 ViT-B/14, Expansion factor=82026.05 | 4 | 50.64 | 82.06 | 89.15 | |
| Matching PursuitBackbone=DINOv2 ViT-B/14, Layer=102026.05 | 4 | 55.81 | 83.89 | 83.5 | |
| Archetypal SAEBackbone=DINOv2 ViT-B/14, Layer=112026.05 | 4 | 41.51 | 81.77 | 71.21 | |
| Matching PursuitBackbone=DINOv2 ViT-B/14, Expansion factor=82026.05 | 3 | 56.92 | 82.86 | 90.28 | |
| Matryoshka SAEBackbone=DINOv2 ViT-B/14, Layer=102026.05 | 3 | 50.63 | 83.61 | 81.15 | |
| Matryoshka SAEBackbone=DINOv2 ViT-B/14, Layer=112026.05 | 3 | 46.55 | 83.61 | 83.84 | |
| Archetypal SAEBackbone=DINOv2 ViT-B/14, Expansion factor=82026.05 | 2 | 51.56 | 81.26 | 84.75 | |
| BatchTopK-SAEBackbone=DINOv2 ViT-B/14, Layer=102026.05 | 2 | 51.58 | 83.61 | 81.66 | |
| BatchTopK-SAEBackbone=DINOv2 ViT-B/14, Layer=112026.05 | 2 | 43.92 | 83.54 | 83.22 | |
| ELUDeBackbone=DINOv2 ViT-B/14, Expansion factor=82026.05 | 1 | 64.85 | 84.29 | 100 | |
| ELUDeBackbone=DINOv2 ViT-B/14, Layer=102026.05 | 1 | 70 | 84.29 | 100 | |
| ELUDeBackbone=DINOv2 ViT-B/14, Layer=112026.05 | 1 | 63.54 | 84.29 | 100 |