Concept Identifiability on Bias-in-Bios
0.9731MCCLinear Probe
Evaluation results
| Method | Links | |
|---|---|---|
| Linear ProbeModel Backbone=Gemma-2-2B2025.02 | 0.9731 | |
| Linear ProbeEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.9456 | |
| SSAEModel Backbone=Gemma-2-2B2025.02 | 0.8232 | |
| SSAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.7924 | |
| JumpReLU SAEModel Backbone=Gemma-2-2B2025.02 | 0.7773 | |
| TopK-SAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.7717 | |
| ReLU-SAEModel Backbone=Gemma-2-2B2025.02 | 0.7657 | |
| PythiaSAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.7524 | |
| GemmaScopeModel Backbone=Gemma-2-2B2025.02 | 0.7385 | |
| ReLU-SAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.6262 | |
| JumpReLU SAEEvaluation Protocol=Recovered decoder directions evaluated on paired observations (f(x), f(x˜))2025.02 | 0.5929 | |
| TopK-SAEModel Backbone=Gemma-2-2B2025.02 | 0.566 |