Causal Faithfulness Evaluation on HaluEval Adversarial
100nAOPCRetrain
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RetrainModel=Pythia-1.4B2026.05 | 100 | 1.855 | 0.0209 | |
| SAEBoostModel=Pythia-1.4B2026.05 | 100 | 1.83 | 0.0166 | |
| FaithfulSAEModel=Pythia-1.4B2026.05 | 100 | 1.678 | 0.0514 | |
| GAEModel=Pythia-1.4B2026.05 | 100 | 1.885 | 0.0163 | |
| FinetuneModel=Pythia-1.4B2026.05 | 98.8 | 1.88 | 0.0174 | |
| FixedModel=Pythia-1.4B2026.05 | 98.5 | 1.425 | 0.0252 | |
| TERMModel=Pythia-1.4B2026.05 | 98.4 | 1.762 | 0.0292 | |
| GAEModel=Pythia-1.4B2026.05 | 96.8 | 1.693 | 0.0276 | |
| SAEBoostModel=Pythia-1.4B2026.05 | 96.5 | 1.53 | 0.0283 | |
| GAEModel=GPT-2 Small2026.05 | 95.3 | 1.303 | 0.0017 | |
| FinetuneModel=GPT-2 Small2026.05 | 93.2 | 1.155 | 0.0218 | |
| FixedModel=GPT-2 Small2026.05 | 93 | 1.134 | 0.0406 | |
| RetrainModel=GPT-2 Small2026.05 | 93 | 1.276 | 0.03 | |
| FinetuneModel=Pythia-1.4B2026.05 | 92.5 | 1.502 | 0.028 | |
| SAEBoostModel=GPT-2 Small2026.05 | 90.9 | 1.243 | 0.0448 | |
| FaithfulSAEModel=GPT-2 Small2026.05 | 90.8 | 1.082 | 0.0278 | |
| FixedModel=Pythia-1.4B2026.05 | 89.9 | 1.021 | 0.0354 | |
| FaithfulSAEModel=Pythia-1.4B2026.05 | 89.9 | 1.171 | 0.0307 | |
| TERMModel=GPT-2 Small2026.05 | 89.8 | 0.987 | 0.0401 | |
| TERMModel=Pythia-1.4B2026.05 | 89.6 | 1.104 | 0.0349 | |
| RetrainModel=Pythia-1.4B2026.05 | 89.4 | 1.393 | 0.0305 | |
| GAEModel=GPT-2 Small2026.05 | 87.1 | 0.963 | 0.0014 | |
| SAEBoostModel=GPT-2 Small2026.05 | 84 | 0.921 | 0.0212 | |
| FaithfulSAEModel=GPT-2 Small2026.05 | 73.8 | 0.74 | 0.0586 | |
| FixedModel=GPT-2 Small2026.05 | 73.5 | 0.737 | 0.0473 | |
| TERMModel=GPT-2 Small2026.05 | 73 | 0.732 | 0.0523 | |
| FinetuneModel=GPT-2 Small2026.05 | 57.9 | 0.579 | 0.0105 | |
| RetrainModel=GPT-2 Small2026.05 | 52.1 | 0.528 | 0.2763 |