Incorrect Reasoning Path Detection on DeepScaleR
64.24AccuracyTokUR (AU)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TokUR (AU)Model=Qwen-2.5-7B-Instruct2025.05 | 64.24 | 76.39 | 71.73 | |
| TokUR (TU)Model=Qwen-2.5-7B-Instruct2025.05 | 63.89 | 76.05 | 71.43 | |
| TokUR (EU)Model=Qwen-2.5-7B-Instruct2025.05 | 61.47 | 73.55 | 68.91 | |
| Self-CertaintyModel=Qwen-2.5-7B-Instruct2025.05 | 56.6 | 66.31 | 58.18 | |
| DeepConfModel=Qwen-2.5-7B-Instruct2025.05 | 55.83 | 64.91 | 56.71 | |
| TokUR (AU)Model=Qwen-2.5-3B-Instruct2025.05 | 54.61 | 77.52 | 67.3 | |
| TokUR (TU)Model=Qwen-2.5-3B-Instruct2025.05 | 53.52 | 76.26 | 65.47 | |
| PEModel=Qwen-2.5-7B-Instruct2025.05 | 52.64 | 60.55 | 50.61 | |
| LLModel=Qwen-2.5-7B-Instruct2025.05 | 52.43 | 60.23 | 50.27 | |
| Self-CertaintyModel=Qwen-2.5-3B-Instruct2025.05 | 52 | 74.22 | 62.55 | |
| DeepConfModel=Qwen-2.5-3B-Instruct2025.05 | 51.93 | 73.85 | 60.75 | |
| TokUR (EU)Model=Qwen-2.5-3B-Instruct2025.05 | 51.35 | 73.33 | 61.08 | |
| PEModel=Qwen-2.5-3B-Instruct2025.05 | 48.09 | 66.57 | 46.44 | |
| LLModel=Qwen-2.5-3B-Instruct2025.05 | 47.37 | 65.66 | 45.43 | |
| CoT (Lower-Bound)Model=Qwen-2.5-7B-Instruct2025.05 | 44.93 | — | — | |
| TokUR (TU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 43.91 | 85.33 | 65.25 | |
| TokUR (AU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 43.89 | 85.31 | 65.2 | |
| TokUR (EU)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 43.89 | 84.92 | 65.57 | |
| Self-CertaintyBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 39.03 | 76.72 | 56.15 | |
| SARBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 37.51 | 73.01 | 42.89 | |
| DeepConfBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 37.48 | 73.05 | 48.76 | |
| CoT (Lower-Bound)Model=Qwen-2.5-3B-Instruct2025.05 | 35.87 | — | — | |
| SEBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 35.55 | 67.68 | 35.18 | |
| INSIDEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 34.13 | 67.05 | 33.83 | |
| UDegBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 29.75 | 59.14 | 32.64 | |
| PEBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 28.65 | 55.9 | 26.8 | |
| LLM-CheckBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 28.37 | 55.42 | 26.46 | |
| LLBackbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 26.85 | 52.82 | 24.48 | |
| TokUR (TU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 25.71 | 83.55 | 47.56 | |
| TokUR (AU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 25.71 | 83.52 | 47.48 | |
| TokUR (EU)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 25.52 | 82.87 | 46.76 | |
| UEccBackbone=Llama-3.1-8B-Instruct, ISO=false2025.05 | 25.48 | 50.16 | 25.08 | |
| CoT (Lower-Bound)Backbone=Llama-3.1-8B-Instruct2025.05 | 24.86 | — | — | |
| DeepConfBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 22 | 71.65 | 29.99 | |
| Self-CertaintyBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 21.76 | 71.93 | 33.81 | |
| P(True)Backbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 17.52 | 59.58 | 17.48 | |
| PEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 17.33 | 56.09 | 14.74 | |
| LLM-CheckBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 17.3 | 55.76 | 14.55 | |
| LLBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 16.83 | 53.84 | 13.93 | |
| UDegBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 16.5 | 56.88 | 18.04 | |
| INSIDEBackbone=Llama-3.2-1B-Instruct, ISO=true2025.05 | 16.3 | 54.73 | 15.5 | |
| P(True)Backbone=Llama-3.1-8B-Instruct, ISO=true2025.05 | 16.23 | 33.64 | 18.06 | |
| CoT (Lower-Bound)Backbone=Llama-3.2-1B-Instruct2025.05 | 14.25 | — | — | |
| UEccBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 14.23 | 48.68 | 13.77 | |
| SEBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 12.58 | 46.3 | 12.94 | |
| SARBackbone=Llama-3.2-1B-Instruct, ISO=false2025.05 | 11.14 | 43.14 | 12.34 |