Science and Engineering Question Answering on TheoremQA
68.04AccuracyMC²
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MC²Backbone=Gemini-2.0-flash, Reasoning Method=MC²2026.04 | 68.04 | 1,753 | |
| MC²Backbone=Qwen3-8B, Reasoning Method=MC²2026.04 | 64.88 | 1,924 | |
| MC²Backbone=Llama-3.1-8B-Instruct, Reasoning Method=MC²2026.04 | 61.46 | 4,899 | |
| ToTBackbone=Gemini-2.0-flash, Reasoning Method=ToT2026.04 | 61.33 | 4,743 | |
| Meta PromptBackbone=Qwen3-8B, Reasoning Method=Meta Prompt2026.04 | 61.29 | 1,448 | |
| CoT-SCBackbone=Gemini-2.0-flash, Reasoning Method=CoT-SC2026.04 | 60.12 | 4,025 | |
| Meta ReasoningBackbone=Gemini-2.0-flash, Reasoning Method=Meta Reasoning2026.04 | 59.54 | 860 | |
| CoTBackbone=Gemini-2.0-flash, Reasoning Method=CoT2026.04 | 59.34 | 813 | |
| Buffer of ThoughtBackbone=Gemini-2.0-flash, Reasoning Method=Buffer of Thought2026.04 | 59.17 | 2,532 | |
| Meta PromptBackbone=Gemini-2.0-flash, Reasoning Method=Meta Prompt2026.04 | 57.5 | 1,832 | |
| ToTBackbone=Qwen3-8B, Reasoning Method=ToT2026.04 | 57.29 | 5,686 | |
| MC²Backbone=GPT-4o-mini, Reasoning Method=MC²2026.04 | 55.96 | 2,159 | |
| CoT-SCBackbone=Qwen3-8B, Reasoning Method=CoT-SC2026.04 | 55.25 | 4,882 | |
| Buffer of ThoughtBackbone=Qwen3-8B, Reasoning Method=Buffer of Thought2026.04 | 54.44 | 2,171 | |
| CoTBackbone=Qwen3-8B, Reasoning Method=CoT2026.04 | 54.33 | 1,022 | |
| Meta PromptBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Meta Prompt2026.04 | 53.79 | 3,709 | |
| TTPIBackbone=Qwen3-8B, Reasoning Method=TTPI2026.04 | 52.29 | 907 | |
| Meta ReasoningBackbone=Qwen3-8B, Reasoning Method=Meta Reasoning2026.04 | 49.92 | 888 | |
| Buffer of ThoughtBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Buffer of Thought2026.04 | 49.92 | 2,662 | |
| Meta ReasoningBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Meta Reasoning2026.04 | 48.8 | 2,052 | |
| CoT-SCBackbone=GPT-4o-mini, Reasoning Method=CoT-SC2026.04 | 46.92 | 3,643 | |
| ToTBackbone=GPT-4o-mini, Reasoning Method=ToT2026.04 | 45.33 | 5,774 | |
| Meta PromptBackbone=GPT-4o-mini, Reasoning Method=Meta Prompt2026.04 | 44.62 | 2,265 | |
| Buffer of ThoughtBackbone=GPT-4o-mini, Reasoning Method=Buffer of Thought2026.04 | 44.42 | 1,867 | |
| TTPIBackbone=GPT-4o-mini, Reasoning Method=TTPI2026.04 | 44.42 | 1,484 | |
| CoTBackbone=GPT-4o-mini, Reasoning Method=CoT2026.04 | 44.04 | 716 | |
| ToTBackbone=Llama-3.1-8B-Instruct, Reasoning Method=ToT2026.04 | 43.79 | 8,296 | |
| Meta ReasoningBackbone=GPT-4o-mini, Reasoning Method=Meta Reasoning2026.04 | 43.62 | 752 | |
| CoT-SCBackbone=Llama-3.1-8B-Instruct, Reasoning Method=CoT-SC2026.04 | 43.08 | 8,010 | |
| CoTBackbone=Llama-3.1-8B-Instruct, Reasoning Method=CoT2026.04 | 39.5 | 1,097 | |
| TTPIBackbone=Llama-3.1-8B-Instruct, Reasoning Method=TTPI2026.04 | 36.29 | 3,757 |