Logic Reasoning on K&K Logic Puzzles In-domain benchmark leaderboard · SOTA2 Research

Logic Reasoning on K&K Logic Puzzles In-domain

98Accuracy (Level 3)o3-mini-high

Evaluation Results

Method	Links
o3-mini-high2025.06		98	97	95	94	89	95
MeRF2025.06		95	92	84	78	66	83
MeRF2025.06		95	94	92	87	71	88
RLVR2025.06		92	89	84	79	64	82
RLVR2025.06		90	82	78	70	55	75
MeRF2025.06		78	73	68	62	42	65
Deepseek-R12025.06		73	77	78	75	88	78
RLVR2025.06		68	67	57	43	22	51
MeRF2025.06		68	62	48	42	20	48
GPT-4o2025.06		57	49	32	23	21	36
MeRF2025.06		54	53	45	36	16	41
RLVR2025.06		52	44	36	18	12	32
o1-2024-12-172025.06		51	38	38	35	30	38
RLVR2025.06		48	53	30	21	16	34
Qwen2.5-14B-Instruct2025.06		46	31	20	9	11	23
GPT-4o-mini2025.06		42	34	17	9	10	22
Qwen2.5-7B-Instruct-1M2025.06		40	25	11	6	2	17
Qwen2.5-7B-Base2025.06		34	16	9	0	0	12
DeepSeek-R1-Distill-Llama-8B2025.06		26	22	14	6	5	15
MeRF2025.06		25	20	12	8	4	14
Qwen2.5-7B-Instruct2025.06		24	10	6	4	4	10
Deepseek-Math-7B2025.06		21	8	6	2	0	7
RLVR2025.06		20	16	16	10	4	13
NuminaMath-7B-CoT2025.06		13	12	5	1	0	6
DeepSeek-R1-Distill-Qwen-1.5B2025.06		8	1	0	0	0	2