Reasoning Quality Assessment on Understanding Fables
0.7191AUROCTRACED
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TRACEDModel=DeepSeek-R1-Llama-8B2026.03 | 0.7191 | 0.6586 | 0.8242 | |
| LR ProbeModel=DeepSeek-R1-Llama-8B2026.03 | 0.7177 | 0.6539 | 0.8297 | |
| CoT-KineticsModel=DeepSeek-R1-Llama-8B2026.03 | 0.7162 | 0.5787 | 0.8627 | |
| TRACEDModel=Qwen3-4B-Thinking-25072026.03 | 0.7088 | 0.6749 | 0.5397 | |
| SAPLMAModel=DeepSeek-R1-Llama-8B2026.03 | 0.6944 | 0.6555 | 0.8919 | |
| SAPLMAModel=Qwen3-4B-Thinking-25072026.03 | 0.6728 | 0.5006 | 0.8876 | |
| PerplexityModel=DeepSeek-R1-Llama-8B2026.03 | 0.6719 | 0.6581 | 0.8568 | |
| TRACEDModel=Llama-3.1-8B-Instruct2026.03 | 0.6676 | 0.629 | 0.6739 | |
| MSPModel=Qwen3-4B-Thinking-25072026.03 | 0.6509 | 0.6727 | 0.8462 | |
| EntropyModel=Qwen3-4B-Thinking-25072026.03 | 0.645 | 0.6549 | 0.8462 | |
| CoEModel=Qwen3-4B-Thinking-25072026.03 | 0.6314 | 0.5108 | 0.7731 | |
| CoT-KineticsModel=Qwen3-4B-Thinking-25072026.03 | 0.6266 | 0.5824 | 0.7313 | |
| TRACEDModel=Qwen2.5-7B-Instruct2026.03 | 0.6238 | 0.738 | 0.806 | |
| MSPModel=Llama-3.1-8B-Instruct2026.03 | 0.6237 | 0.6159 | 0.8786 | |
| SAPLMAModel=Qwen2.5-7B-Instruct2026.03 | 0.6221 | 0.6818 | 0.875 | |
| LR ProbeModel=Qwen3-4B-Thinking-25072026.03 | 0.6167 | 0.4667 | 0.75 | |
| EntropyModel=DeepSeek-R1-Llama-8B2026.03 | 0.6156 | 0.6481 | 0.8378 | |
| LR ProbeModel=Qwen2.5-7B-Instruct2026.03 | 0.6109 | 0.734 | 0.8125 | |
| EntropyModel=Llama-3.1-8B-Instruct2026.03 | 0.6086 | 0.6065 | 0.8355 | |
| MSPModel=DeepSeek-R1-Llama-8B2026.03 | 0.6044 | 0.6427 | 0.8919 | |
| LR ProbeModel=Llama-3.1-8B-Instruct2026.03 | 0.5995 | 0.5567 | 0.7419 | |
| CoEModel=DeepSeek-R1-Llama-8B2026.03 | 0.5856 | 0.5603 | 0.8649 | |
| PerplexityModel=Llama-3.1-8B-Instruct2026.03 | 0.5839 | 0.5915 | 0.8677 | |
| SAPLMAModel=Llama-3.1-8B-Instruct2026.03 | 0.572 | 0.5285 | 0.871 | |
| CoT-KineticsModel=Qwen2.5-7B-Instruct2026.03 | 0.5674 | 0.6079 | 0.8311 | |
| CoEModel=Llama-3.1-8B-Instruct2026.03 | 0.5516 | 0.5681 | 0.8677 | |
| PerplexityModel=Qwen3-4B-Thinking-25072026.03 | 0.5503 | 0.6344 | 0.8675 | |
| CoEModel=Qwen2.5-7B-Instruct2026.03 | 0.5283 | 0.5724 | 0.8688 | |
| CoT-KineticsModel=Llama-3.1-8B-Instruct2026.03 | 0.5269 | 0.4793 | 0.8535 | |
| MSPModel=Qwen2.5-7B-Instruct2026.03 | 0.46 | 0.5457 | 0.8564 | |
| EntropyModel=Qwen2.5-7B-Instruct2026.03 | 0.4385 | 0.5329 | 0.8375 | |
| PerplexityModel=Qwen2.5-7B-Instruct2026.03 | 0.4185 | 0.5178 | 0.8532 |