List your company
ResearchFormerly WizWand

Chain-of-Thought Attribution Faithfulness on GSM8K (test)

232.39AUPCAttriCoT-2x
42.340491.6802141.02190.3598Jun 20, 2026
Updated 4mo ago

Evaluation results

MethodLinks
AttriCoT-2xCoT Model=DS-Llama-8B, Forward Pass Count=2(S + T)2026.06232.39
AttriCoTCoT Model=DS-Llama-8B, Forward Pass Count=S + T2026.06231
AttriCoT-2xCoT Model=DS-Qwen-14B, Forward Pass Count=2(S + T)2026.06218.55
AttriCoTCoT Model=DS-Qwen-14B, Forward Pass Count=S + T2026.06216.7
TA-KL (no log)CoT Model=DS-Llama-8B, Log Transformation=false2026.06195.61
TA-KLCoT Model=DS-Llama-8B, Log Transformation=true2026.06183.29
TA-KL (no log)CoT Model=DS-Qwen-14B, Log Transformation=false2026.06181.82
TA-KLCoT Model=DS-Qwen-14B, Log Transformation=true2026.06169.47
AttriCoT-2xCoT Model=Qwen3-8B, Forward Pass Count=2(S + T)2026.06168.03
AttriCoTCoT Model=Qwen3-8B, Forward Pass Count=S + T2026.06166.59
TA-KL (no log)CoT Model=Qwen3-8B, Log Transformation=false2026.06161.28
Prompted-ICL (counterpart)CoT Model=DS-Qwen-14B, Prompting Strategy=counterpart, In-Context Learning (ICL)=true2026.06153.51
Prompted (counterpart)CoT Model=DS-Qwen-14B, Prompting Strategy=counterpart, In-Context Learning (ICL)=false2026.06147.27
Prompted-ICL (GPT-OSS)CoT Model=DS-Qwen-14B, Prompting Strategy=GPT-OSS-120B, In-Context Learning (ICL)=true2026.06142.58
AttriCoT-2xCoT Model=DS-Qwen3-8B, Forward Pass Count=2(S + T)2026.06139.73
AttriCoTCoT Model=DS-Qwen3-8B, Forward Pass Count=S + T2026.06138.64
Prompted-ICL (GPT-OSS)CoT Model=DS-Llama-8B, Prompting Strategy=GPT-OSS-120B, In-Context Learning (ICL)=true2026.06135.79
TA-KL (no log)CoT Model=DS-Qwen3-8B, Log Transformation=false2026.06123.42
Prompted-ICL (self)CoT Model=DS-Qwen-14B, Prompting Strategy=self, In-Context Learning (ICL)=true2026.06117.77
TA-KLCoT Model=DS-Qwen3-8B, Log Transformation=true2026.06116.05
Attention-based (trained)CoT Model=DS-Qwen-14B, trained=true2026.06111.39
Prompted-ICL (self)CoT Model=DS-Llama-8B, Prompting Strategy=self, In-Context Learning (ICL)=true2026.06107.29
Attention-basedCoT Model=DS-Llama-8B, trained=false2026.06107.06
Attention-based (trained)CoT Model=DS-Llama-8B, trained=true2026.06101.43
TA-KLCoT Model=Qwen3-8B, Log Transformation=true2026.0698.53
Prompted (GPT-OSS)CoT Model=DS-Llama-8B, Prompting Strategy=GPT-OSS-120B, In-Context Learning (ICL)=false2026.0696.23
Prompted (self)CoT Model=DS-Qwen-14B, Prompting Strategy=self, In-Context Learning (ICL)=false2026.0695.52
Prompted (GPT-OSS)CoT Model=DS-Qwen-14B, Prompting Strategy=GPT-OSS-120B, In-Context Learning (ICL)=false2026.0694.87
Prompted (self)CoT Model=DS-Llama-8B, Prompting Strategy=self, In-Context Learning (ICL)=false2026.0688.65
Attention-basedCoT Model=DS-Qwen-14B, trained=false2026.0687.36
Prompted-ICL (GPT-OSS)CoT Model=Qwen3-8B, Prompting Strategy=GPT-OSS-120B, In-Context Learning (ICL)=true2026.0672.06
Prompted (self)CoT Model=Qwen3-8B, Prompting Strategy=self, In-Context Learning (ICL)=false2026.0667.45
Attention-basedCoT Model=DS-Qwen3-8B, trained=false2026.0665.72
Prompted-ICL (GPT-OSS)CoT Model=DS-Qwen3-8B, Prompting Strategy=GPT-OSS-120B, In-Context Learning (ICL)=true2026.0662.06
Attention-based (trained)CoT Model=DS-Qwen3-8B, trained=true2026.0661.88
Prompted (self)CoT Model=DS-Qwen3-8B, Prompting Strategy=self, In-Context Learning (ICL)=false2026.0661.39
Prompted-ICL (self)CoT Model=DS-Qwen3-8B, Prompting Strategy=self, In-Context Learning (ICL)=true2026.0660.38
Prompted (counterpart)CoT Model=DS-Qwen3-8B, Prompting Strategy=counterpart, In-Context Learning (ICL)=false2026.0660.2
Prompted-ICL (counterpart)CoT Model=DS-Qwen3-8B, Prompting Strategy=counterpart, In-Context Learning (ICL)=true2026.0658.39
Attention-basedCoT Model=Qwen3-8B, trained=false2026.0658.31
Attention-based (trained)CoT Model=Qwen3-8B, trained=true2026.0656.02
Prompted (GPT-OSS)CoT Model=Qwen3-8B, Prompting Strategy=GPT-OSS-120B, In-Context Learning (ICL)=false2026.0654.78
Prompted-ICL (self)CoT Model=Qwen3-8B, Prompting Strategy=self, In-Context Learning (ICL)=true2026.0650.49
Prompted (GPT-OSS)CoT Model=DS-Qwen3-8B, Prompting Strategy=GPT-OSS-120B, In-Context Learning (ICL)=false2026.0649.65