List your company
ResearchFormerly WizWand

Incorrect Reasoning Path Detection on MATH500

94AccuracyTokUR (TU)
18.152837.843957.53577.2261May 16, 2025
Updated 6mo ago

Evaluation results

MethodLinks
TokUR (TU)Model=Qwen-2.5-7B-Instruct2025.059484.6193.43
TokUR (AU)Model=Qwen-2.5-7B-Instruct2025.059484.7493.65
TokUR (EU)Model=Qwen-2.5-7B-Instruct2025.059281.8791.68
TokUR (AU)Model=Qwen-2.5-3B-Instruct2025.0587.3382.5588.3
TokUR (TU)Model=Qwen-2.5-3B-Instruct2025.0586.6781.4887.42
Self-CertaintyModel=Qwen-2.5-7B-Instruct2025.0585.669.5486.24
DeepConfModel=Qwen-2.5-7B-Instruct2025.0584.5366.8683.85
TokUR (EU)Model=Qwen-2.5-3B-Instruct2025.0584.2778.9685.37
LLModel=Qwen-2.5-7B-Instruct2025.0582.1362.1980.19
PEModel=Qwen-2.5-7B-Instruct2025.058262.5180.29
Self-CertaintyModel=Qwen-2.5-3B-Instruct2025.0581.3374.9584.76
DeepConfModel=Qwen-2.5-3B-Instruct2025.0580.5374.3983.81
PEModel=Qwen-2.5-3B-Instruct2025.0576.868.9477.3
LLModel=Qwen-2.5-3B-Instruct2025.0576.1368.4276.82
CoT (Lower-Bound)Model=Qwen-2.5-7B-Instruct2025.0575.07——
TokUR (TU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.057482.4779.62
TokUR (AU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.057482.4379.56
TokUR (EU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.0572.482.8681.35
Self-CertaintyBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0569.0776.4176.22
DeepConfBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0566.2771.8669.57
CoT (Lower-Bound)Model=Qwen-2.5-3B-Instruct2025.0564.73——
SARBackbone=Llama-3.1-8B-Instruct, ISO=false2025.0559.269.4263.74
INSIDEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0557.3362.9455.06
SEBackbone=Llama-3.1-8B-Instruct, ISO=false2025.0555.7362.9355.21
UDegBackbone=Llama-3.1-8B-Instruct, ISO=false2025.0553.4758.6257.69
PEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0553.0757.9849.72
LLM-CheckBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0552.857.4149.69
LLBackbone=Llama-3.1-8B-Instruct, ISO=true2025.0551.0755.3647.24
UEccBackbone=Llama-3.1-8B-Instruct, ISO=false2025.0549.650.2349.48
CoT (Lower-Bound)Backbone=Llama-3.1-8B-Instruct2025.0548.6——
TokUR (TU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.0544.6780.6456.79
TokUR (AU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.0544.6780.6156.73
TokUR (EU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.0544.1379.7456.64
DeepConfBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0539.8771.7746
Self-CertaintyBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0538.1371.1748.37
P(True)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.0535.3333.4136.05
LLM-CheckBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0531.3356.4127.01
PEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0531.3357.0826.88
UDegBackbone=Llama-3.2-1B-Instruct, ISO=false2025.0530.9360.5736.32
LLBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0529.8755.4125.88
INSIDEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.0529.255.7128.82
P(True)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.0527.654.3826.39
CoT (Lower-Bound)Backbone=Llama-3.2-1B-Instruct2025.0525.6——
UEccBackbone=Llama-3.2-1B-Instruct, ISO=false2025.0525.248.7525.79
SEBackbone=Llama-3.2-1B-Instruct, ISO=false2025.0524.1347.2925.71
SARBackbone=Llama-3.2-1B-Instruct, ISO=false2025.0521.0744.5724.03