Reasoning Quality Assessment on GSM8K
80.61AUROCTRACED
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TRACEDModel=DeepSeek-R1-Llama-8B2026.03 | 80.61 | 82.83 | 65 | |
| SAPLMAModel=DeepSeek-R1-Llama-8B2026.03 | 79.96 | 82.04 | 68.24 | |
| LR ProbeModel=DeepSeek-R1-Llama-8B2026.03 | 79.95 | 81.95 | 70.59 | |
| TRACEDModel=Qwen3-4B-Thinking-25072026.03 | 78.25 | 77.58 | 57 | |
| LR ProbeModel=Qwen3-4B-Thinking-25072026.03 | 77.72 | 71.76 | 57.54 | |
| TRACEDModel=Llama-3.1-8B-Instruct2026.03 | 75.56 | 77.14 | 77.5 | |
| SAPLMAModel=Qwen3-4B-Thinking-25072026.03 | 75.1 | 72.4 | 59.62 | |
| CoEModel=Llama-3.1-8B-Instruct2026.03 | 74.71 | 76.7 | 86.43 | |
| MSPModel=DeepSeek-R1-Llama-8B2026.03 | 74.24 | 74.25 | 82.35 | |
| CoT-KineticsModel=Qwen3-4B-Thinking-25072026.03 | 74.17 | 60 | 75.1 | |
| CoEModel=Qwen3-4B-Thinking-25072026.03 | 72.93 | 77.4 | 78.46 | |
| SAPLMAModel=Llama-3.1-8B-Instruct2026.03 | 70.11 | 73.92 | 77.67 | |
| LR ProbeModel=Llama-3.1-8B-Instruct2026.03 | 69.66 | 71.99 | 83.33 | |
| TRACEDModel=Qwen2.5-7B-Instruct2026.03 | 69.56 | 68.95 | 80.24 | |
| EntropyModel=DeepSeek-R1-Llama-8B2026.03 | 68.63 | 69.16 | 82.35 | |
| CoT-KineticsModel=Qwen2.5-7B-Instruct2026.03 | 68.29 | 68.77 | 86 | |
| SAPLMAModel=Qwen2.5-7B-Instruct2026.03 | 68.1 | 68.7 | 80.93 | |
| LR ProbeModel=Qwen2.5-7B-Instruct2026.03 | 67.21 | 68.45 | 81.38 | |
| MSPModel=Qwen3-4B-Thinking-25072026.03 | 65.09 | 64.89 | 86.54 | |
| EntropyModel=Qwen2.5-7B-Instruct2026.03 | 64.95 | 56.16 | 81.36 | |
| MSPModel=Qwen2.5-7B-Instruct2026.03 | 64.89 | 56.59 | 83.1 | |
| EntropyModel=Qwen3-4B-Thinking-25072026.03 | 64.35 | 64.06 | 86.54 | |
| CoEModel=Qwen2.5-7B-Instruct2026.03 | 62.36 | 44.76 | 84.83 | |
| CoT-KineticsModel=DeepSeek-R1-Llama-8B2026.03 | 61.94 | 55.27 | 83.26 | |
| PerplexityModel=Qwen3-4B-Thinking-25072026.03 | 61.91 | 59.17 | 80.77 | |
| PerplexityModel=DeepSeek-R1-Llama-8B2026.03 | 60.96 | 60.88 | 88.24 | |
| CoEModel=DeepSeek-R1-Llama-8B2026.03 | 56.51 | 56.53 | 88.24 | |
| MSPModel=Llama-3.1-8B-Instruct2026.03 | 47.7 | 57.17 | 88.21 | |
| EntropyModel=Llama-3.1-8B-Instruct2026.03 | 45.75 | 52.76 | 83.42 | |
| PerplexityModel=Llama-3.1-8B-Instruct2026.03 | 44.83 | 49.56 | 85.46 | |
| PerplexityModel=Qwen2.5-7B-Instruct2026.03 | 37.72 | 41.23 | 86.55 | |
| CoT-KineticsModel=Llama-3.1-8B-Instruct2026.03 | 32.76 | 48.32 | 85.34 |