# Latent Concept Detection on AxBench full 500 concepts Source: [Latent Concept Detection on AxBench full 500 concepts]() Collection index: [llms.txt]() Data updated: 2026-05-15T08:09:22.649Z Evaluation of model performance on the AxBench full 500 concepts dataset for the task of Latent Concept Detection, measured by Mean AUROC, EPMean, and EPExemplar. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Latent Concept Detection | | Dataset (including any stated split) | AxBench full 500 concepts | | Metric | Mean AUROC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 13 of 13 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Mean AUROC](): Higher is better; unit: PERCENT. - [EPMean](): Higher is better; unit: PERCENT. - [EPExemplar](): Higher is better; unit: PERCENT. ## Recorded results | Method | Mean AUROC | Result date | Report paper | | --- | --- | --- | --- | | ReFT-r1 | 96.5 | Not recorded | 6a06a8a7dbc836b87d82a17a | | DiffMean | 94.6 | Not recorded | 6a06a8a7dbc836b87d82a17a | | Probe | 94.6 | Not recorded | 6a06a8a7dbc836b87d82a17a | | BoW | 93.1 | Not recorded | 6a06a8a7dbc836b87d82a17a | | Prompt | 92.1 | Not recorded | 6a06a8a7dbc836b87d82a17a | | SAE-A | 91.1 | Not recorded | 6a06a8a7dbc836b87d82a17a | | LAT | 80.9 | Not recorded | 6a06a8a7dbc836b87d82a17a | | SAE | 75.5 | Not recorded | 6a06a8a7dbc836b87d82a17a | | PCA | 71.2 | Not recorded | 6a06a8a7dbc836b87d82a17a | ### ReFT-r1 Mean AUROC: 96.5. Result ID: 6a06d4a747f34ce924ee54f6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Supervision":"labels at construction","Backbone":"Gemma-2-2B-it L20"}. This record does not assert independent reproduction. ### DiffMean Mean AUROC: 94.6. Result ID: 6a06d4a747f34ce924ee54f7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Supervision":"labels at construction","Backbone":"Gemma-2-2B-it L20"}. This record does not assert independent reproduction. ### Probe Mean AUROC: 94.6. Result ID: 6a06d4a747f34ce924ee54f8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Supervision":"labels at construction","Backbone":"Gemma-2-2B-it L20"}. This record does not assert independent reproduction. ### BoW Mean AUROC: 93.1. Result ID: 6a06d4a747f34ce924ee54f9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Supervision":"labels at construction","Backbone":"Gemma-2-2B-it L20"}. This record does not assert independent reproduction. ### Prompt Mean AUROC: 92.1. Result ID: 6a06d4a747f34ce924ee54fa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Supervision":"concept text as prompt","Backbone":"Gemma-2-2B-it L20"}. This record does not assert independent reproduction. ### SAE-A Mean AUROC: 91.1. Result ID: 6a06d4a747f34ce924ee54fb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Supervision":"labels at selection","K (Region count)":"16,384","Backbone":"Gemma-2-2B-it L20"}. This record does not assert independent reproduction. ### LAT Mean AUROC: 80.9. Result ID: 6a06d4a747f34ce924ee54fc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Supervision":"labels at construction, pos. only","Backbone":"Gemma-2-2B-it L20"}. This record does not assert independent reproduction. ### SAE Mean AUROC: 75.5. Result ID: 6a06d4a747f34ce924ee54fd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Supervision":"paired feature, no selection","K (Region count)":"16,384","Backbone":"Gemma-2-2B-it L20"}. This record does not assert independent reproduction. ### PCA Mean AUROC: 71.2. Result ID: 6a06d4a747f34ce924ee54fe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Supervision":"labels at construction, pos. only","Backbone":"Gemma-2-2B-it L20"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.