Concept Identifiability on TruthfulQA
0.7758MCCLinear Probe
Evaluation results
| Method | Links | |
|---|---|---|
| Linear ProbeModel Backbone=Gemma-2-2B2025.02 | 0.7758 | |
| SSAEModel Backbone=Gemma-2-2B2025.02 | 0.7585 | |
| GemmaScopeModel Backbone=Gemma-2-2B2025.02 | 0.737 | |
| TopK-SAEModel Backbone=Gemma-2-2B2025.02 | 0.7281 | |
| Linear ProbeEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.7144 | |
| ReLU-SAEModel Backbone=Gemma-2-2B2025.02 | 0.7128 | |
| JumpReLU SAEModel Backbone=Gemma-2-2B2025.02 | 0.7128 | |
| SSAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.6952 | |
| ReLU-SAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.6634 | |
| JumpReLU SAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.6484 | |
| TopK-SAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.6234 | |
| PythiaSAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.6226 |