Logical Reasoning on Game-of-24
94.33AccuracyMC²
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MC²Backbone=Gemini-2.0-flash, Reasoning Method=MC²2026.04 | 94.33 | 1,219 | |
| MC²Backbone=GPT-4o-mini, Reasoning Method=MC²2026.04 | 88.67 | 2,479 | |
| Meta PromptBackbone=Gemini-2.0-flash, Reasoning Method=Meta Prompt2026.04 | 88 | 1,592 | |
| ToTBackbone=Gemini-2.0-flash, Reasoning Method=ToT2026.04 | 83.33 | 5,549 | |
| Meta ReasoningBackbone=Gemini-2.0-flash, Reasoning Method=Meta Reasoning2026.04 | 81 | 1,682 | |
| CoT-SCBackbone=Gemini-2.0-flash, Reasoning Method=CoT-SC2026.04 | 80.67 | 3,085 | |
| MC²Backbone=Qwen3-8B, Reasoning Method=MC²2026.04 | 80.67 | 4,445 | |
| Buffer of ThoughtBackbone=Gemini-2.0-flash, Reasoning Method=Buffer of Thought2026.04 | 78 | 2,158 | |
| CoTBackbone=Gemini-2.0-flash, Reasoning Method=CoT2026.04 | 72 | 599 | |
| CoT-SCBackbone=Qwen3-8B, Reasoning Method=CoT-SC2026.04 | 68.67 | 7,325 | |
| ToTBackbone=Qwen3-8B, Reasoning Method=ToT2026.04 | 61.67 | 6,562 | |
| TTPIBackbone=Qwen3-8B, Reasoning Method=TTPI2026.04 | 58.67 | 6,410 | |
| CoTBackbone=Qwen3-8B, Reasoning Method=CoT2026.04 | 48 | 2,960 | |
| Meta PromptBackbone=Qwen3-8B, Reasoning Method=Meta Prompt2026.04 | 48 | 7,393 | |
| Meta ReasoningBackbone=Qwen3-8B, Reasoning Method=Meta Reasoning2026.04 | 47.33 | 2,818 | |
| Buffer of ThoughtBackbone=Qwen3-8B, Reasoning Method=Buffer of Thought2026.04 | 45.67 | 2,880 | |
| MC²Backbone=Llama-3.1-8B-Instruct, Reasoning Method=MC²2026.04 | 45.33 | 12,808 | |
| Meta PromptBackbone=GPT-4o-mini, Reasoning Method=Meta Prompt2026.04 | 41.33 | 3,178 | |
| Meta PromptBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Meta Prompt2026.04 | 41 | 22,895 | |
| ToTBackbone=GPT-4o-mini, Reasoning Method=ToT2026.04 | 33.33 | 3,801 | |
| ToTBackbone=Llama-3.1-8B-Instruct, Reasoning Method=ToT2026.04 | 33 | 15,249 | |
| TTPIBackbone=Llama-3.1-8B-Instruct, Reasoning Method=TTPI2026.04 | 33 | 15,106 | |
| Meta ReasoningBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Meta Reasoning2026.04 | 32.67 | 12,631 | |
| Buffer of ThoughtBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Buffer of Thought2026.04 | 28.67 | 6,146 | |
| TTPIBackbone=GPT-4o-mini, Reasoning Method=TTPI2026.04 | 28 | 916 | |
| CoT-SCBackbone=GPT-4o-mini, Reasoning Method=CoT-SC2026.04 | 27 | 2,461 | |
| CoT-SCBackbone=Llama-3.1-8B-Instruct, Reasoning Method=CoT-SC2026.04 | 24 | 30,741 | |
| Meta ReasoningBackbone=GPT-4o-mini, Reasoning Method=Meta Reasoning2026.04 | 20.33 | 964 | |
| Buffer of ThoughtBackbone=GPT-4o-mini, Reasoning Method=Buffer of Thought2026.04 | 19.33 | 1,539 | |
| CoTBackbone=Llama-3.1-8B-Instruct, Reasoning Method=CoT2026.04 | 16.67 | 14,734 | |
| CoTBackbone=GPT-4o-mini, Reasoning Method=CoT2026.04 | 7.67 | 553 |