Incorrect Reasoning Path Detection on GSM8K
98.31AccuracyDeepConf
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DeepConfModel=Qwen-2.5-7B-Instruct2025.05 | 98.31 | 81.94 | 97.88 | |
| TokUR (AU)Model=Qwen-2.5-7B-Instruct2025.05 | 97.79 | 80.7 | 97.44 | |
| Self-CertaintyModel=Qwen-2.5-7B-Instruct2025.05 | 97.69 | 80.92 | 97.81 | |
| TokUR (TU)Model=Qwen-2.5-7B-Instruct2025.05 | 97.69 | 80.15 | 97.36 | |
| PEModel=Qwen-2.5-7B-Instruct2025.05 | 97.23 | 77.4 | 97.35 | |
| TokUR (EU)Model=Qwen-2.5-7B-Instruct2025.05 | 97.23 | 74.95 | 96.61 | |
| LLModel=Qwen-2.5-7B-Instruct2025.05 | 97.08 | 75.37 | 97.02 | |
| DeepConfBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 96.56 | 83.3 | 96.23 | |
| Self-CertaintyBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 96.26 | 80.6 | 95.65 | |
| TokUR (AU)Model=Qwen-2.5-3B-Instruct2025.05 | 96.15 | 78.81 | 95.76 | |
| TokUR (TU)Model=Qwen-2.5-3B-Instruct2025.05 | 96.05 | 77.66 | 95.46 | |
| DeepConfModel=Qwen-2.5-3B-Instruct2025.05 | 95.69 | 78.06 | 95.63 | |
| TokUR (TU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 95.54 | 81.01 | 95.53 | |
| TokUR (AU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 95.49 | 80.97 | 95.52 | |
| LLModel=Qwen-2.5-3B-Instruct2025.05 | 94.92 | 75.54 | 94.93 | |
| Self-CertaintyModel=Qwen-2.5-3B-Instruct2025.05 | 94.77 | 77.22 | 95.66 | |
| TokUR (EU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 94.67 | 78.31 | 94.91 | |
| PEModel=Qwen-2.5-3B-Instruct2025.05 | 94.46 | 76.44 | 95.06 | |
| TokUR (EU)Model=Qwen-2.5-3B-Instruct2025.05 | 94.26 | 70.75 | 93.6 | |
| LLM-CheckBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 93.23 | 73.98 | 93.37 | |
| PEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 93.23 | 74.03 | 93.37 | |
| UDegBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 92.62 | 67.22 | 92.24 | |
| LLBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 92.46 | 72.21 | 92.64 | |
| CoT (Lower-Bound)Model=Qwen-2.5-7B-Instruct2025.05 | 92.15 | — | — | |
| INSIDEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 88.21 | 58.86 | 87.44 | |
| SARBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 87.99 | 60.16 | 89.24 | |
| CoT (Lower-Bound)Model=Qwen-2.5-3B-Instruct2025.05 | 87.72 | — | — | |
| SEBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 86.77 | 55.61 | 87.16 | |
| CoT (Lower-Bound)Backbone=Llama-3.1-8B-Instruct2025.05 | 85.69 | — | — | |
| UEccBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 84.87 | 47.47 | 84.69 | |
| P(True)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 82.77 | 41.94 | 82.19 | |
| UDegBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 71.99 | 66.6 | 75.72 | |
| DeepConfBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 62.77 | 75.7 | 69.72 | |
| TokUR (TU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 62.31 | 75.07 | 70.29 | |
| TokUR (AU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 62.21 | 75.03 | 70.22 | |
| Self-CertaintyBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 61.38 | 73.41 | 68.38 | |
| PEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 59.85 | 71.21 | 61.61 | |
| TokUR (EU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 59.74 | 71.79 | 66.4 | |
| UEccBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 59.62 | 49.05 | 60.02 | |
| LLM-CheckBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 59.54 | 71.01 | 61.29 | |
| LLBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 57.38 | 69.01 | 58.51 | |
| P(True)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 48.92 | 56.64 | 48.22 | |
| INSIDEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 45.79 | 53.66 | 46.03 | |
| CoT (Lower-Bound)Backbone=Llama-3.2-1B-Instruct2025.05 | 44.43 | — | — | |
| SARBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 43.95 | 50.28 | 43.24 | |
| SEBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 42.62 | 50.64 | 45.09 |