Latent Concept Detection on AxBench full 500 concepts
96.5Mean AUROCReFT-r1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ReFT-r1Supervision=labels at construction, Backbone=Gemma-2-2B-it L202026.05 | 96.5 | — | — | |
| DiffMeanSupervision=labels at construction, Backbone=Gemma-2-2B-it L202026.05 | 94.6 | — | — | |
| ProbeSupervision=labels at construction, Backbone=Gemma-2-2B-it L202026.05 | 94.6 | — | — | |
| BoWSupervision=labels at construction, Backbone=Gemma-2-2B-it L202026.05 | 93.1 | — | — | |
| PromptSupervision=concept text as prompt, Backbone=Gemma-2-2B-it L202026.05 | 92.1 | — | — | |
| SAE-ASupervision=labels at selection, K (Region count)=16,384, Backbone=Gemma-2-2B-it L202026.05 | 91.1 | — | — | |
| LATSupervision=labels at construction, pos. only, Backbone=Gemma-2-2B-it L202026.05 | 80.9 | — | — | |
| SAESupervision=paired feature, no selection, K (Region count)=16,384, Backbone=Gemma-2-2B-it L202026.05 | 75.5 | — | — | |
| PCASupervision=labels at construction, pos. only, Backbone=Gemma-2-2B-it L202026.05 | 71.2 | — | — | |
| EP (p1)K (Region count)=5,796, Percentile Resolution (p)=p1, Backbone=Gemma-2-2B-it L202026.05 | — | 86.4 | 88.1 | |
| EP (p2)K (Region count)=2,037, Percentile Resolution (p)=p2, Backbone=Gemma-2-2B-it L202026.05 | — | 85.7 | 85 | |
| EP (p4)K (Region count)=686, Percentile Resolution (p)=p4, Backbone=Gemma-2-2B-it L202026.05 | — | 84.7 | 86.9 | |
| EP (p8)K (Region count)=226, Percentile Resolution (p)=p8, Backbone=Gemma-2-2B-it L202026.05 | — | 81 | 75.5 |