List your company
ResearchFormerly WizWand

Incorrect Reasoning Path Detection on GSM8K

98.31AccuracyDeepConf
40.392455.428770.46585.5013May 16, 2025
Updated 6mo ago

Evaluation results

MethodLinks
DeepConfModel=Qwen-2.5-7B-Instruct2025.0598.3181.9497.88
TokUR (AU)Model=Qwen-2.5-7B-Instruct2025.0597.7980.797.44
Self-CertaintyModel=Qwen-2.5-7B-Instruct2025.0597.6980.9297.81
TokUR (TU)Model=Qwen-2.5-7B-Instruct2025.0597.6980.1597.36
PEModel=Qwen-2.5-7B-Instruct2025.0597.2377.497.35
TokUR (EU)Model=Qwen-2.5-7B-Instruct2025.0597.2374.9596.61
LLModel=Qwen-2.5-7B-Instruct2025.0597.0875.3797.02
DeepConfBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0596.5683.396.23
Self-CertaintyBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0596.2680.695.65
TokUR (AU)Model=Qwen-2.5-3B-Instruct2025.0596.1578.8195.76
TokUR (TU)Model=Qwen-2.5-3B-Instruct2025.0596.0577.6695.46
DeepConfModel=Qwen-2.5-3B-Instruct2025.0595.6978.0695.63
TokUR (TU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.0595.5481.0195.53
TokUR (AU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.0595.4980.9795.52
LLModel=Qwen-2.5-3B-Instruct2025.0594.9275.5494.93
Self-CertaintyModel=Qwen-2.5-3B-Instruct2025.0594.7777.2295.66
TokUR (EU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.0594.6778.3194.91
PEModel=Qwen-2.5-3B-Instruct2025.0594.4676.4495.06
TokUR (EU)Model=Qwen-2.5-3B-Instruct2025.0594.2670.7593.6
LLM-CheckBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0593.2373.9893.37
PEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0593.2374.0393.37
UDegBackbone=Llama-3.1-8B-Instruct, ISO=false2025.0592.6267.2292.24
LLBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0592.4672.2192.64
CoT (Lower-Bound)Model=Qwen-2.5-7B-Instruct2025.0592.15——
INSIDEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0588.2158.8687.44
SARBackbone=Llama-3.1-8B-Instruct, ISO=false2025.0587.9960.1689.24
CoT (Lower-Bound)Model=Qwen-2.5-3B-Instruct2025.0587.72——
SEBackbone=Llama-3.1-8B-Instruct, ISO=false2025.0586.7755.6187.16
CoT (Lower-Bound)Backbone=Llama-3.1-8B-Instruct2025.0585.69——
UEccBackbone=Llama-3.1-8B-Instruct, ISO=false2025.0584.8747.4784.69
P(True)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.0582.7741.9482.19
UDegBackbone=Llama-3.2-1B-Instruct, ISO=false2025.0571.9966.675.72
DeepConfBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0562.7775.769.72
TokUR (TU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.0562.3175.0770.29
TokUR (AU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.0562.2175.0370.22
Self-CertaintyBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0561.3873.4168.38
PEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0559.8571.2161.61
TokUR (EU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.0559.7471.7966.4
UEccBackbone=Llama-3.2-1B-Instruct, ISO=false2025.0559.6249.0560.02
LLM-CheckBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0559.5471.0161.29
LLBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0557.3869.0158.51
P(True)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.0548.9256.6448.22
INSIDEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0545.7953.6646.03
CoT (Lower-Bound)Backbone=Llama-3.2-1B-Instruct2025.0544.43——
SARBackbone=Llama-3.2-1B-Instruct, ISO=false2025.0543.9550.2843.24
SEBackbone=Llama-3.2-1B-Instruct, ISO=false2025.0542.6250.6445.09