CoT faithfulness detection on HLE Bio
78AccuracyCIE-SCORER
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CIE-SCORERParadigm=Circuit-based2026.05 | 78 | 79.7 | |
| BiGGenParadigm=LLM-as-judge2026.05 | 70.2 | 69.2 | |
| Answer TracingParadigm=Logits-based2026.05 | 64.3 | 76.2 | |
| Information GainParadigm=Logits-based2026.05 | 52.5 | 9.5 | |
| PerplexityParadigm=Baselines2026.05 | 47.4 | 52.4 | |
| Adding MistakesParadigm=Counterfactual-based2026.05 | 46.4 | 51.6 | |
| Early AnsweringParadigm=Counterfactual-based2026.05 | 46.4 | 48.3 | |
| ParaphrasingParadigm=Counterfactual-based2026.05 | 45.7 | 40 | |
| Option ShufflingParadigm=Counterfactual-based2026.05 | 40 | 14.3 | |
| Removing StepsParadigm=Counterfactual-based2026.05 | 39.3 | 37 | |
| RandomParadigm=Baselines2026.05 | 35.7 | 43.8 |