Logic Reasoning on K&K Logic Puzzles OOD
99Score Threshold 2 (OOD)o3-mini-high
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| o3-mini-high2025.06 | 99 | 83 | 94 | |
| MeRFBackbone=Qwen2.5-14B-Instruct2025.06 | 99 | 65 | 83 | |
| MeRFBackbone=DeepSeek-R1-Distill-Llama-8B2025.06 | 99 | 64 | 86 | |
| Deepseek-R12025.06 | 91 | 83 | 81 | |
| RLVRBackbone=Qwen2.5-14B-Instruct2025.06 | 90 | 42 | 72 | |
| RLVRBackbone=DeepSeek-R1-Distill-Llama-8B2025.06 | 90 | 58 | 79 | |
| o1-2024-12-172025.06 | 83 | 20 | 42 | |
| MeRFBackbone=Qwen2.5-7B-Instruct2025.06 | 76 | 39 | 63 | |
| MeRFBackbone=Qwen2.5-7B-Instruct-1M2025.06 | 76 | 25 | 49 | |
| RLVRBackbone=Qwen2.5-7B-Instruct2025.06 | 71 | 28 | 51 | |
| MeRFBackbone=Qwen2.5-7B-Base2025.06 | 70 | 18 | 42 | |
| GPT-4o2025.06 | 68 | 11 | 37 | |
| GPT-4o-mini2025.06 | 63 | 1 | 25 | |
| Qwen2.5-14B-Instruct2025.06 | 63 | 6 | 27 | |
| RLVRBackbone=Qwen2.5-7B-Instruct-1M2025.06 | 60 | 16 | 34 | |
| RLVRBackbone=Qwen2.5-7B-Base2025.06 | 59 | 17 | 35 | |
| Qwen2.5-7B-Instruct-1M2025.06 | 49 | 1 | 19 | |
| Qwen2.5-7B-Instruct2025.06 | 43 | 0 | 13 | |
| MeRFBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.06 | 43 | 4 | 17 | |
| Qwen2.5-7B-Base2025.06 | 41 | 0 | 14 | |
| Deepseek-Math-7B2025.06 | 35 | 0 | 10 | |
| DeepSeek-R1-Distill-Qwen-1.5B2025.06 | 30 | 0 | 6 | |
| RLVRBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.06 | 29 | 1 | 14 | |
| NuminaMath-7B-CoT2025.06 | 28 | 0 | 8 | |
| DeepSeek-R1-Distill-Llama-8B2025.06 | 24 | 11 | 15 |