Causal Faithfulness Evaluation on FineWeb Temporal
0.985nAOPCGAE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GAEModel=Pythia-1.4B2026.05 | 0.985 | 1.677 | 0.0207 | |
| RetrainModel=Pythia-1.4B2026.05 | 0.983 | 1.681 | 0.0563 | |
| FaithfulSAEModel=Pythia-1.4B2026.05 | 0.982 | 1.642 | 0.0307 | |
| SAEBoostModel=Pythia-1.4B2026.05 | 0.971 | 1.451 | 0.0211 | |
| TERMModel=Pythia-1.4B2026.05 | 0.965 | 1.486 | 0.0237 | |
| FinetuneModel=Pythia-1.4B2026.05 | 0.963 | 1.618 | 0.0216 | |
| FixedModel=Pythia-1.4B2026.05 | 0.962 | 1.536 | 0.0216 | |
| GAEModel=GPT-2 Small2026.05 | 0.96 | 1.494 | 0.0167 | |
| SAEBoostModel=GPT-2 Small2026.05 | 0.958 | 1.476 | 0.0177 | |
| FaithfulSAEModel=GPT-2 Small2026.05 | 0.936 | 1.186 | 0.0213 | |
| GAEModel=Pythia-1.4B2026.05 | 0.915 | 1.354 | 0.0269 | |
| SAEBoostModel=Pythia-1.4B2026.05 | 0.908 | 1.296 | 0.0284 | |
| RetrainModel=GPT-2 Small2026.05 | 0.895 | 1.118 | 0.0218 | |
| RetrainModel=Pythia-1.4B2026.05 | 0.894 | 1.315 | 0.0405 | |
| FinetuneModel=Pythia-1.4B2026.05 | 0.859 | 1.249 | 0.0264 | |
| FaithfulSAEModel=Pythia-1.4B2026.05 | 0.858 | 1.056 | 0.0272 | |
| FixedModel=GPT-2 Small2026.05 | 0.857 | 1.017 | 0.0281 | |
| FinetuneModel=GPT-2 Small2026.05 | 0.856 | 0.984 | 0.0172 | |
| TERMModel=GPT-2 Small2026.05 | 0.853 | 0.993 | 0.0283 | |
| TERMModel=Pythia-1.4B2026.05 | 0.845 | 1.043 | 0.0271 | |
| FixedModel=Pythia-1.4B2026.05 | 0.839 | 1.091 | 0.0278 | |
| GAEModel=GPT-2 Small2026.05 | 0.768 | 0.871 | 0.0011 | |
| RetrainModel=GPT-2 Small2026.05 | 0.766 | 0.856 | 0.0375 | |
| TERMModel=GPT-2 Small2026.05 | 0.741 | 0.79 | 0.0183 | |
| FixedModel=GPT-2 Small2026.05 | 0.735 | 0.796 | 0.0185 | |
| FinetuneModel=GPT-2 Small2026.05 | 0.725 | 0.802 | 0.0015 | |
| FaithfulSAEModel=GPT-2 Small2026.05 | 0.725 | 0.76 | 0.0262 | |
| SAEBoostModel=GPT-2 Small2026.05 | 0.704 | 0.786 | 0.012 |