Reasoning Quality Assessment on GPQA
0.83AUROCTRACED
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TRACEDModel=DeepSeek-R1-Llama-8B2026.03 | 0.83 | 0.6607 | 0.64 | |
| TRACEDModel=Qwen2.5-7B-Instruct2026.03 | 0.7636 | 0.7583 | 0.475 | |
| SAPLMAModel=Qwen2.5-7B-Instruct2026.03 | 0.7622 | 0.745 | 0.4257 | |
| LR ProbeModel=Qwen3-4B-Thinking-25072026.03 | 0.76 | 0.7683 | 0.6 | |
| LR ProbeModel=DeepSeek-R1-Llama-8B2026.03 | 0.7588 | 0.5451 | 0.8571 | |
| LR ProbeModel=Llama-3.1-8B-Instruct2026.03 | 0.7571 | 0.6552 | 0.5 | |
| TRACEDModel=Llama-3.1-8B-Instruct2026.03 | 0.7344 | 0.6794 | 0.525 | |
| SAPLMAModel=Llama-3.1-8B-Instruct2026.03 | 0.7333 | 0.6661 | 0.4286 | |
| LR ProbeModel=Qwen2.5-7B-Instruct2026.03 | 0.7194 | 0.668 | 0.5383 | |
| SAPLMAModel=DeepSeek-R1-Llama-8B2026.03 | 0.718 | 0.5505 | 0.7143 | |
| TRACEDModel=Qwen3-4B-Thinking-25072026.03 | 0.705 | 0.7328 | 0.425 | |
| CoEModel=Qwen2.5-7B-Instruct2026.03 | 0.6833 | 0.645 | 0.6889 | |
| EntropyModel=Qwen2.5-7B-Instruct2026.03 | 0.6806 | 0.7038 | 0.7778 | |
| SAPLMAModel=Qwen3-4B-Thinking-25072026.03 | 0.68 | 0.6962 | 0.4402 | |
| MSPModel=Qwen2.5-7B-Instruct2026.03 | 0.6389 | 0.6519 | 0.7778 | |
| MSPModel=Qwen3-4B-Thinking-25072026.03 | 0.6 | 0.6787 | 0.8 | |
| CoT-KineticsModel=Qwen3-4B-Thinking-25072026.03 | 0.58 | 0.6 | 0.85 | |
| PerplexityModel=Qwen3-4B-Thinking-25072026.03 | 0.56 | 0.545 | 0.6 | |
| EntropyModel=Qwen3-4B-Thinking-25072026.03 | 0.56 | 0.6587 | 0.8 | |
| CoT-KineticsModel=DeepSeek-R1-Llama-8B2026.03 | 0.549 | 0.5 | 0.85 | |
| CoEModel=Qwen3-4B-Thinking-25072026.03 | 0.54 | 0.6746 | 0.8576 | |
| CoT-KineticsModel=Llama-3.1-8B-Instruct2026.03 | 0.5238 | 0.5399 | 0.8714 | |
| CoEModel=DeepSeek-R1-Llama-8B2026.03 | 0.5 | 0.6 | 0.8317 | |
| EntropyModel=DeepSeek-R1-Llama-8B2026.03 | 0.4857 | 0.5159 | 0.8823 | |
| CoT-KineticsModel=Qwen2.5-7B-Instruct2026.03 | 0.4722 | 0.6 | 0.7444 | |
| PerplexityModel=Qwen2.5-7B-Instruct2026.03 | 0.3917 | 0.3871 | 0.8723 | |
| MSPModel=Llama-3.1-8B-Instruct2026.03 | 0.3857 | 0.4917 | 0.8678 | |
| MSPModel=DeepSeek-R1-Llama-8B2026.03 | 0.3837 | 0.475 | 0.8887 | |
| PerplexityModel=DeepSeek-R1-Llama-8B2026.03 | 0.3831 | 0.4107 | 0.8767 | |
| CoEModel=Llama-3.1-8B-Instruct2026.03 | 0.381 | 0.5403 | 0.8325 | |
| EntropyModel=Llama-3.1-8B-Instruct2026.03 | 0.3186 | 0.3959 | 0.8324 | |
| PerplexityModel=Llama-3.1-8B-Instruct2026.03 | 0.3143 | 0.3709 | 0.8672 |