Faithfulness evaluation on TellMeWhy
2.25AUC π-Soft-NSHETA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HETAModel=LLaMA-3.1 70B2026.04 | 2.25 | 8.6 | |
| HETAModel=Phi-3-Medium-14B2026.04 | 2.2 | 9.5 | |
| HETABackbone=Qwen2.5 3B2026.04 | 2.1 | 9 | |
| HETAModel=GPT-J 6B2026.04 | 2.04 | 9.2 | |
| ContextCiteModel=LLaMA-3.1 70B2026.04 | 0.56 | 1.2 | |
| Peering (PML)Backbone=Qwen2.5 3B2026.04 | 0.5 | 1.8 | |
| ContextCiteBackbone=Qwen2.5 3B2026.04 | 0.47 | 1.65 | |
| ReAgentBackbone=Qwen2.5 3B2026.04 | 0.39 | 1.47 | |
| ReAgentModel=Phi-3-Medium-14B2026.04 | 0.39 | 1.47 | |
| Grad-ELLMBackbone=Mistral2026.01 | 0.368 | 0.531 | |
| Grad-ELLMBackbone=Mistral2026.01 | 0.368 | 0.531 | |
| ReAgentModel=GPT-J 6B2026.04 | 0.36 | 1.45 | |
| ReAgentModel=LLaMA-3.1 70B2026.04 | 0.34 | 1.28 | |
| RandomBackbone=Mistral2026.01 | 0.335 | 0.564 | |
| RandomBackbone=Mistral2026.01 | 0.335 | 0.564 | |
| SEA-CoTBackbone=Qwen2.5 3B2026.04 | 0.33 | 1.32 | |
| SEA-CoTModel=Phi-3-Medium-14B2026.04 | 0.33 | 1.32 | |
| DeepLIFTBackbone=Llama2026.01 | 0.313 | 1.034 | |
| SEA-CoTModel=GPT-J 6B2026.04 | 0.31 | 1.3 | |
| SaliencyBackbone=Llama2026.01 | 0.309 | 0.796 | |
| Grad-ELLMBackbone=Llama2026.01 | 0.308 | 1.322 | |
| Input×GradientsBackbone=Mistral2026.01 | 0.308 | 0.462 | |
| Layer Gradient×ActivationBackbone=Mistral2026.01 | 0.308 | 0.46 | |
| Input×GradientsBackbone=Mistral2026.01 | 0.308 | 0.462 | |
| Layer Gradient×ActivationBackbone=Mistral2026.01 | 0.308 | 0.46 | |
| DeepLIFTBackbone=Mistral2026.01 | 0.307 | 0.467 | |
| DeepLIFTBackbone=Mistral2026.01 | 0.307 | 0.467 | |
| SaliencyBackbone=Mistral2026.01 | 0.304 | 0.465 | |
| SaliencyBackbone=Mistral2026.01 | 0.304 | 0.465 | |
| Input×GradientsBackbone=Llama2026.01 | 0.299 | 0.745 | |
| Layer Gradient×ActivationBackbone=Llama2026.01 | 0.299 | 0.758 | |
| Integrated GradientsBackbone=Llama2026.01 | 0.286 | 1.356 | |
| RandomBackbone=Llama2026.01 | 0.283 | 0.796 | |
| SEA-CoTModel=LLaMA-3.1 70B2026.04 | 0.28 | 1.15 | |
| Progressive InferenceBackbone=Qwen2.5 3B2026.04 | 0.26 | 1.18 | |
| Progressive InferenceModel=Phi-3-Medium-14B2026.04 | 0.26 | 1.18 | |
| Integrated GradientsBackbone=Mistral2026.01 | 0.257 | 0.386 | |
| Integrated GradientsBackbone=Mistral2026.01 | 0.257 | 0.386 | |
| Progressive InferenceModel=GPT-J 6B2026.04 | 0.25 | 1.12 | |
| AttentionBackbone=Mistral2026.01 | 0.247 | 0.426 | |
| AttentionBackbone=Mistral2026.01 | 0.247 | 0.426 | |
| AttentionBackbone=Llama2026.01 | 0.225 | 0.684 | |
| Progressive InferenceModel=LLaMA-3.1 70B2026.04 | 0.22 | 1 | |
| Value ZeroingBackbone=Mistral2026.01 | 0.19 | 0.389 | |
| Value ZeroingBackbone=Mistral2026.01 | 0.19 | 0.389 | |
| Value ZeroingBackbone=Llama2026.01 | 0.162 | 0.398 | |
| Peering (PML)Model=LLaMA-3.1 70B2026.04 | 0.12 | 0.15 | |
| Integrated GradientsModel=LLaMA-3.1 70B2026.04 | 0.1 | 0.13 | |
| Integrated GradientsBackbone=Qwen2.5 3B2026.04 | 0.09 | 1.3 | |
| Peering (PML)Model=Phi-3-Medium-14B2026.04 | 0.08 | 1.75 | |
| Peering (PML)Model=GPT-J 6B2026.04 | 0.06 | 1.68 | |
| Integrated GradientsModel=Phi-3-Medium-14B2026.04 | 0.06 | 1.6 | |
| Integrated GradientsModel=GPT-J 6B2026.04 | 0.04 | 1.54 | |
| Attention RolloutModel=LLaMA-3.1 70B2026.04 | 0.01 | -1.48 | |
| TDD-backwardBackbone=Qwen2.5 3B2026.04 | 0 | 1.82 | |
| TDD-backwardModel=Phi-3-Medium-14B2026.04 | -0.02 | 1.82 | |
| TDD-backwardModel=GPT-J 6B2026.04 | -0.03 | 1.89 | |
| Attention RolloutBackbone=Qwen2.5 3B2026.04 | -0.07 | 0.22 | |
| Attention RolloutModel=Phi-3-Medium-14B2026.04 | -0.08 | 0.3 | |
| Attention RolloutModel=GPT-J 6B2026.04 | -0.09 | 0.25 | |
| fAMLModel=GPT-J 6B2026.04 | -0.09 | 0.05 | |
| fAMLBackbone=Qwen2.5 3B2026.04 | -0.1 | 0.08 | |
| fAMLModel=Phi-3-Medium-14B2026.04 | -0.1 | 0.08 | |
| TDD-backwardModel=LLaMA-3.1 70B2026.04 | -0.1 | 0.01 | |
| ContextCiteModel=Phi-3-Medium-14B2026.04 | -0.2 | 1.45 | |
| fAMLModel=LLaMA-3.1 70B2026.04 | -0.21 | 0.46 | |
| ContextCiteModel=GPT-J 6B2026.04 | -0.22 | 1.46 |