Concept Faithfulness Evaluation on AGNEWS
89.95Perturbed AccuracySAE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SAEModel=Llama2025.06 | 89.95 | -0.1 | |
| SAEModel=Qwen2025.06 | 89.92 | -0.2 | |
| CLVQ-VAEModel=Llama2025.06 | 87.58 | 1.6 | |
| Single-LayerModel=Llama2025.06 | 87.44 | 1.8 | |
| ClusteringModel=Llama2025.06 | 84.85 | 4.7 | |
| ClusteringModel=Qwen2025.06 | 75.42 | 15 | |
| CLVQ-VAEModel=Qwen2025.06 | 75.36 | 15.1 | |
| Single-LayerModel=Qwen2025.06 | 71.64 | 19.3 | |
| Single-LayerModel=BERT2025.06 | 67.11 | 10 | |
| ClusteringModel=BERT2025.06 | 66.75 | 10.5 | |
| CLVQ-VAEModel=BERT2025.06 | 64.92 | 13 | |
| ClusteringModel=RoBERTa2025.06 | 38.75 | 46.7 | |
| SAEModel=BERT2025.06 | 33.44 | 55.2 | |
| SAEModel=RoBERTa2025.06 | 32.8 | 54.9 | |
| Single-LayerModel=RoBERTa2025.06 | 10.11 | 86.1 | |
| CLVQ-VAEModel=RoBERTa2025.06 | 9.92 | 86.4 |