Reasoning Quality Assessment on SocialIQA
77.94AUROCTRACED
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TRACEDModel=Qwen2.5-7B-Instruct2026.03 | 77.94 | 73.71 | 77.33 | — | |
| TRACEDBase Model=DeepSeek-R12026.03 | 75.4 | — | — | 0.004 | |
| TRACEDModel=DeepSeek-R1-Llama-8B2026.03 | 75.36 | 69.09 | 65 | — | |
| CoEModel=Qwen2.5-7B-Instruct2026.03 | 74.86 | 73.62 | 77.42 | — | |
| SAPLMAModel=Qwen2.5-7B-Instruct2026.03 | 73.87 | 70.13 | 78.46 | — | |
| TRACEDModel=Llama-3.1-8B-Instruct2026.03 | 72.13 | 62.33 | 70 | — | |
| TRACEDModel=Qwen3-4B-Thinking-25072026.03 | 71.94 | 66.58 | 53.75 | — | |
| LR ProbeBase Model=DeepSeek-R1, Method Status=Runner-up Baseline2026.03 | 71 | — | — | — | |
| LR ProbeModel=DeepSeek-R1-Llama-8B2026.03 | 70.97 | 68.83 | 70.91 | — | |
| LR ProbeModel=Qwen2.5-7B-Instruct2026.03 | 70.81 | 65.57 | 77.5 | — | |
| CoT-KineticsModel=Qwen2.5-7B-Instruct2026.03 | 70.8 | 62.18 | 81.21 | — | |
| SAPLMAModel=DeepSeek-R1-Llama-8B2026.03 | 69.57 | 68.91 | 70.97 | — | |
| LR ProbeModel=Qwen3-4B-Thinking-25072026.03 | 68.21 | 64.43 | 57.68 | — | |
| MSPModel=Qwen3-4B-Thinking-25072026.03 | 67.41 | 64.41 | 80.36 | — | |
| CoT-KineticsModel=DeepSeek-R1-Llama-8B2026.03 | 67.38 | 60.1 | 84.88 | — | |
| EntropyModel=Qwen3-4B-Thinking-25072026.03 | 66.74 | 64.85 | 83.93 | — | |
| SAPLMAModel=Llama-3.1-8B-Instruct2026.03 | 66.4 | 61.74 | 86.08 | — | |
| EntropyModel=Qwen2.5-7B-Instruct2026.03 | 64.92 | 61.19 | 84.63 | — | |
| CoEModel=DeepSeek-R1-Llama-8B2026.03 | 64.65 | 63.38 | 82.26 | — | |
| CoEModel=Qwen3-4B-Thinking-25072026.03 | 64.48 | 65.26 | 83.93 | — | |
| LR ProbeModel=Llama-3.1-8B-Instruct2026.03 | 64.45 | 62.25 | 84.31 | — | |
| MSPModel=Qwen2.5-7B-Instruct2026.03 | 64.07 | 60.69 | 86.15 | — | |
| SAPLMAModel=Qwen3-4B-Thinking-25072026.03 | 63.04 | 58.8 | 76.79 | — | |
| CoT-KineticsModel=Llama-3.1-8B-Instruct2026.03 | 62.36 | 51.56 | 84.37 | — | |
| CoEModel=Llama-3.1-8B-Instruct2026.03 | 62.33 | 53.16 | 83.24 | — | |
| PerplexityModel=Qwen3-4B-Thinking-25072026.03 | 61.59 | 60.48 | 87.5 | — | |
| EntropyModel=DeepSeek-R1-Llama-8B2026.03 | 59.25 | 60.95 | 81.94 | — | |
| PerplexityModel=DeepSeek-R1-Llama-8B2026.03 | 59.11 | 61.74 | 80.65 | — | |
| PerplexityModel=Qwen2.5-7B-Instruct2026.03 | 58.76 | 57.96 | 81.54 | — | |
| MSPModel=DeepSeek-R1-Llama-8B2026.03 | 58.67 | 62.37 | 91.94 | — | |
| PerplexityModel=Llama-3.1-8B-Instruct2026.03 | 49.33 | 52.29 | 87.44 | — | |
| EntropyModel=Llama-3.1-8B-Instruct2026.03 | 48.29 | 52.2 | 86.82 | — | |
| MSPModel=Llama-3.1-8B-Instruct2026.03 | 48.17 | 51.64 | 89.13 | — | |
| CoT-KineticsModel=Qwen3-4B-Thinking-25072026.03 | 30.71 | 50 | 85 | — |