Science Reasoning on GPQA (test)
64.44AccuracyInjectRLOpt
Evaluation Results
| Method | Links | |
|---|---|---|
| InjectRLOptbackbone=Qwen3-32B2026.02 | 64.44 | |
| InjectRLOptbackbone=Qwen3-14B2026.02 | 63.33 | |
| InjectCorrectbackbone=Qwen3-32B2026.02 | 62.32 | |
| Baselinebackbone=Qwen3-32B2026.02 | 60.61 | |
| InjectCorrectbackbone=Qwen3-14B2026.02 | 60.51 | |
| Baselinebackbone=Qwen3-14B2026.02 | 60.4 | |
| InjectRLOptbackbone=Qwen3-8B2026.02 | 59.49 | |
| Baselinebackbone=Qwen3-8B2026.02 | 58.69 | |
| InjectCorrectbackbone=Qwen3-8B2026.02 | 57.07 | |
| InjectCorrectbackbone=Qwen3-4B2026.02 | 54.14 | |
| InjectRLOptbackbone=Qwen3-4B2026.02 | 53.23 | |
| Baselinebackbone=Qwen3-4B2026.02 | 52.22 | |
| MANGOBase LLM=GPT-4o-mini2026.05 | 40.4 | |
| MaASBase LLM=GPT-4o-mini2026.05 | 39.39 | |
| MANGO (Skip-2)Base LLM=GPT-4o-mini2026.05 | 38.89 | |
| GPTSwarmBase LLM=GPT-4o-mini2026.05 | 38.38 | |
| AFlowBase LLM=GPT-4o-mini2026.05 | 38.38 | |
| AgentSquareBase LLM=GPT-4o-mini2026.05 | 37.88 | |
| MAmmoTH2-8x7B-PlusParameter Size=8x7B, Training Stage=Continue trained with additional instruction datasets, Few-shot CoT=5-shot2024.05 | 37.8 | |
| Plan-and-ExecuteBase LLM=GPT-4o-mini2026.05 | 37.37 | |
| DyLANBase LLM=GPT-4o-mini2026.05 | 36.87 | |
| MAmmoTH2-8x7BParameter Size=8x7B, Training Stage=Trained only with WEBINSTRUCT, Few-shot CoT=5-shot2024.05 | 36.8 | |
| MAmmoTH2-7B-PlusParameter Size=7B, Training Stage=Continue trained with additional instruction datasets, Few-shot CoT=5-shot2024.05 | 36.8 | |
| GeniusTraining Corpus=OpenHermes2.5 (32K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 34.82 | |
| ADASBase LLM=GPT-4o-mini2026.05 | 34.57 | |
| Self-RefineBase LLM=GPT-4o-mini2026.05 | 31.82 | |
| MultiPersonaBase LLM=GPT-4o-mini2026.05 | 31.82 | |
| MAmmoTH2-34BParameter Size=34B, Training Stage=Trained only with WEBINSTRUCT, Few-shot CoT=5-shot2024.05 | 31.8 | |
| LLM-DebateBase LLM=GPT-4o-mini2026.05 | 31.4 | |
| SC (CoT×5)Base LLM=GPT-4o-mini2026.05 | 30.81 | |
| GeniusTraining Corpus=Magpie (25K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 30.35 | |
| Self-RewardingTraining Corpus=OpenHermes2.5 (32K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 30.13 | |
| SPINTraining Corpus=Magpie (25K), Supervision Level=With Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 29.91 | |
| CoHTraining Corpus=OpenHermes2.5 (32K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 29.69 | |
| CoTBase LLM=GPT-4o-mini2026.05 | 29.29 | |
| SCPOTraining Corpus=OpenHermes2.5 (32K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 28.79 | |
| IO (direct LLM invocation)Base LLM=GPT-4o-mini2026.05 | 28.79 | |
| SCPOTraining Corpus=Magpie (25K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 28.57 | |
| SPINTraining Corpus=OpenHermes2.5 (32K), Supervision Level=With Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 28.57 | |
| Self-RewardingTraining Corpus=Magpie (25K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 28.35 | |
| CoHTraining Corpus=Magpie (25K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 28.13 | |
| SFTTraining Corpus=Magpie (25K), Supervision Level=With Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 26.79 | |
| LLaMA3.1-8B-InstructReasoning Strategy=CoT, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 26.56 | |
| SFTTraining Corpus=OpenHermes2.5 (32K), Supervision Level=With Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 23.44 | |
| STaRTraining Corpus=OpenHermes2.5 (32K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 22.99 | |
| MemGenModel Backbone=Qwen 3 4B Instruct, Inference Strategy=MemGen2026.01 | 21.68 | |
| FlashMemModel Backbone=Qwen 3 4B Instruct, Inference Strategy=FlashMem2026.01 | 20.54 | |
| STaRTraining Corpus=Magpie (25K), Supervision Level=Without Supervision, Backbone Model=LLaMA3.1-8B-Instruct2025.04 | 20.31 | |
| CoT-SCModel Backbone=Qwen 3 4B Instruct, Inference Strategy=CoT-SC2026.01 | 17.95 | |
| VanillaModel Backbone=Qwen 3 4B Instruct, Inference Strategy=Vanilla2026.01 | 16.84 | |
| MemGenModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=MemGen2026.01 | 16.13 | |
| CoT-SCModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=CoT-SC2026.01 | 15.14 | |
| FlashMemModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=FlashMem2026.01 | 14.73 | |
| SnapKVModel Backbone=Qwen 3 4B Instruct, Inference Strategy=SnapKV2026.01 | 14.54 | |
| VanillaModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=Vanilla2026.01 | 13.96 | |
| SnapKVModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=SnapKV2026.01 | 13.66 |