Logic Puzzles on K&K Logic Puzzles
95Accuracy (Level 3)GRPO-SG
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GRPO-SGBackbone=Qwen2.5-7B-Instruct-1M2025.10 | 95 | 95 | 92 | 87 | 84 | 91 | |
| GRPOBackbone=Qwen2.5-7B-Instruct-1M2025.10 | 92 | 90 | 74 | 59 | 60 | 75 | |
| GRPOBackbone=LLaMA3.1-8B-Instruct2025.10 | 92 | 92 | 83 | 79 | 80 | 85 | |
| GRPO-SGBackbone=LLaMA3.1-8B-Instruct2025.10 | 89 | 95 | 88 | 87 | 81 | 88 | |
| GRPO-SGBackbone=Qwen2.5-3B-Instruct2025.10 | 76 | 76 | 61 | 59 | 44 | 63 | |
| GRPO-SGBackbone=Qwen2.5-3B-Base2025.10 | 68 | 62 | 44 | 47 | 26 | 49 | |
| GRPOBackbone=Qwen2.5-3B-Instruct2025.10 | 63 | 47 | 32 | 37 | 29 | 42 | |
| GRPOBackbone=Qwen2.5-3B-Base2025.10 | 60 | 54 | 43 | 38 | 28 | 45 | |
| GRPO-SGBackbone=Mistral-7B-Instruct-v0.32025.10 | 47 | 27 | 18 | 15 | 8 | 23 | |
| GRPOBackbone=Mistral-7B-Instruct-v0.32025.10 | 29 | 16 | 9 | 11 | 3 | 14 | |
| Qwen2.5-7B-Instruct-1MModel Type=Instruct-1M2025.10 | 24 | 18 | 11 | 10 | 4 | 13 | |
| Qwen2.5-3B-BaseModel Type=Base2025.10 | 14 | 4 | 2 | 1 | 2 | 5 | |
| Qwen2.5-3B-InstructModel Type=Instruct2025.10 | 10 | 10 | 7 | 6 | 2 | 7 | |
| LLaMA3.1-8B-InstructModel Type=Instruct2025.10 | 8 | 0 | 0 | 0 | 0 | 2 | |
| Mistral-7B-Instruct-v0.3Model Type=Instruct2025.10 | 5 | 1 | 0 | 2 | 0 | 2 |