Logical Reasoning on K&K Logic Puzzles
96Accuracy (Level 3)Lopti
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LoptiBackbone=Qwen2.5-7B-Instruct-1M2025.10 | 96 | 92 | 86 | 81 | 82 | 87 | 16 | |
| ARBackbone=Qwen2.5-7B-Instruct-1M2025.10 | 95 | 97 | 89 | 84 | 80 | 89 | 18.7 | |
| GRPO-SGBackbone=Qwen2.5-7B-Instruct-1M2025.10 | 95 | 95 | 92 | 87 | 84 | 91 | 21.3 | |
| 80/20Backbone=Qwen2.5-7B-Instruct-1M2025.10 | 94 | 93 | 84 | 78 | 74 | 85 | 13.3 | |
| GRPOBackbone=Qwen2.5-7B-Instruct-1M2025.10 | 92 | 90 | 74 | 59 | 60 | 75 | — | |
| GRPO-SGBackbone=Qwen2.5-3B-Instruct2025.10 | 76 | 76 | 61 | 59 | 44 | 63 | 50 | |
| LoptiBackbone=Qwen2.5-3B-Instruct2025.10 | 70 | 69 | 57 | 46 | 34 | 55 | 31 | |
| 80/20Backbone=Qwen2.5-3B-Instruct2025.10 | 67 | 58 | 53 | 47 | 27 | 50 | 19 | |
| ARBackbone=Qwen2.5-3B-Instruct2025.10 | 65 | 60 | 54 | 44 | 36 | 52 | 23.8 | |
| GRPOBackbone=Qwen2.5-3B-Instruct2025.10 | 63 | 47 | 32 | 37 | 29 | 42 | — | |
| Qwen2.5-7B-Instruct-1MBackbone=Qwen2.5-7B-Instruct-1M2025.10 | 24 | 18 | 11 | 10 | 4 | 13 | — | |
| Qwen2.5-3B-InstructBackbone=Qwen2.5-3B-Instruct2025.10 | 10 | 10 | 7 | 6 | 2 | 7 | — |