Faithfulness Evaluation on Halogen benchmark leaderboard · SOTA2 Research

Faithfulness Evaluation on Halogen

73CODEDistributional Semantics Tracing

Evaluation Results

Method
Distributional Semantics Tracing2025.10	73	77	83	84	78	79
Sparse Autoencoders2025.10	64	53	61	52	63	59
Distributional Semantics Tracing2025.10	63	59	67	77	65	66
Patchscopes2025.10	62	68	45	64	49	58
Subsequence Analysis Tracing2025.10	57	45	56	55	59	54
Causal Path Tracing2025.10	56	54	65	54	49	56
Causal Path Tracing2025.10	56	54	60	65	56	58
Sparse Autoencoders2025.10	54	49	51	44	55	51
Token Evolution (Logit Lens)2025.10	54	43	45	45	54	48
Subsequence Analysis Tracing2025.10	54	59	56	57	59	57
Patchscopes2025.10	49	56	53	45	54	51
Token Evolution (Logit Lens)2025.10	45	52	44	54	39	47
attention2025.10	45	29	31	21	22	30
Reagent2025.10	42	44	43	45	33	41
Reagent2025.10	35	45	19	24	19	29
lime2025.10	34	33	45	29	29	34
gradient-shap2025.10	32	40	35	34	20	32
gradient-shap2025.10	31	35	13	10	15	21
lime2025.10	29	32	15	13	14	21
attention2025.10	24	28	19	12	12	19