Incorrect Reasoning Path Detection on MATH500
94AccuracyTokUR (TU)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| TokUR (TU)Model=Qwen-2.5-7B-Instruct2025.05 | 94 | 84.61 | 93.43 | |
| TokUR (AU)Model=Qwen-2.5-7B-Instruct2025.05 | 94 | 84.74 | 93.65 | |
| TokUR (EU)Model=Qwen-2.5-7B-Instruct2025.05 | 92 | 81.87 | 91.68 | |
| TokUR (AU)Model=Qwen-2.5-3B-Instruct2025.05 | 87.33 | 82.55 | 88.3 | |
| TokUR (TU)Model=Qwen-2.5-3B-Instruct2025.05 | 86.67 | 81.48 | 87.42 | |
| Self-CertaintyModel=Qwen-2.5-7B-Instruct2025.05 | 85.6 | 69.54 | 86.24 | |
| DeepConfModel=Qwen-2.5-7B-Instruct2025.05 | 84.53 | 66.86 | 83.85 | |
| TokUR (EU)Model=Qwen-2.5-3B-Instruct2025.05 | 84.27 | 78.96 | 85.37 | |
| LLModel=Qwen-2.5-7B-Instruct2025.05 | 82.13 | 62.19 | 80.19 | |
| PEModel=Qwen-2.5-7B-Instruct2025.05 | 82 | 62.51 | 80.29 | |
| Self-CertaintyModel=Qwen-2.5-3B-Instruct2025.05 | 81.33 | 74.95 | 84.76 | |
| DeepConfModel=Qwen-2.5-3B-Instruct2025.05 | 80.53 | 74.39 | 83.81 | |
| PEModel=Qwen-2.5-3B-Instruct2025.05 | 76.8 | 68.94 | 77.3 | |
| LLModel=Qwen-2.5-3B-Instruct2025.05 | 76.13 | 68.42 | 76.82 | |
| CoT (Lower-Bound)Model=Qwen-2.5-7B-Instruct2025.05 | 75.07 | — | — | |
| TokUR (TU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 74 | 82.47 | 79.62 | |
| TokUR (AU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 74 | 82.43 | 79.56 | |
| TokUR (EU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 72.4 | 82.86 | 81.35 | |
| Self-CertaintyBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 69.07 | 76.41 | 76.22 | |
| DeepConfBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 66.27 | 71.86 | 69.57 | |
| CoT (Lower-Bound)Model=Qwen-2.5-3B-Instruct2025.05 | 64.73 | — | — | |
| SARBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 59.2 | 69.42 | 63.74 | |
| INSIDEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 57.33 | 62.94 | 55.06 | |
| SEBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 55.73 | 62.93 | 55.21 | |
| UDegBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 53.47 | 58.62 | 57.69 | |
| PEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 53.07 | 57.98 | 49.72 | |
| LLM-CheckBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 52.8 | 57.41 | 49.69 | |
| LLBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 51.07 | 55.36 | 47.24 | |
| UEccBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 49.6 | 50.23 | 49.48 | |
| CoT (Lower-Bound)Backbone=Llama-3.1-8B-Instruct2025.05 | 48.6 | — | — | |
| TokUR (TU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 44.67 | 80.64 | 56.79 | |
| TokUR (AU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 44.67 | 80.61 | 56.73 | |
| TokUR (EU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 44.13 | 79.74 | 56.64 | |
| DeepConfBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 39.87 | 71.77 | 46 | |
| Self-CertaintyBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 38.13 | 71.17 | 48.37 | |
| P(True)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 35.33 | 33.41 | 36.05 | |
| LLM-CheckBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 31.33 | 56.41 | 27.01 | |
| PEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 31.33 | 57.08 | 26.88 | |
| UDegBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 30.93 | 60.57 | 36.32 | |
| LLBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 29.87 | 55.41 | 25.88 | |
| INSIDEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 29.2 | 55.71 | 28.82 | |
| P(True)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 27.6 | 54.38 | 26.39 | |
| CoT (Lower-Bound)Backbone=Llama-3.2-1B-Instruct2025.05 | 25.6 | — | — | |
| UEccBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 25.2 | 48.75 | 25.79 | |
| SEBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 24.13 | 47.29 | 25.71 | |
| SARBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 21.07 | 44.57 | 24.03 |