Mathematical Reasoning on AIME 25 (Acc@8, Pass@8)
86.7AccuracyHermes@1
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Hermes@1Model=OpenAI o3-mini2025.11 | 86.7 | — | — | — | |
| Hermes@5+MajorityModel=OpenAI o3-mini2025.11 | 86.7 | — | — | — | |
| Hermes@5+SkyworkModel=OpenAI o3-mini2025.11 | 86.7 | — | — | — | |
| CoT@5+SkyworkModel=OpenAI o3-mini2025.11 | 83.3 | — | — | — | |
| CoT@5+SafeModel=OpenAI o3-mini2025.11 | 83.3 | — | — | — | |
| CoT@5+Safe*Model=OpenAI o3-mini2025.11 | 83.3 | — | — | — | |
| CoT@5+ShepherdModel=OpenAI o3-mini2025.11 | 80 | — | — | — | |
| Hermes@5+SkyworkModel=DeepSeek-V3.22025.11 | 80 | — | — | — | |
| CoT@5+ArmoRMModel=OpenAI o3-mini2025.11 | 76.7 | — | — | — | |
| Hermes@5+MajorityModel=DeepSeek-V3.22025.11 | 76.7 | — | — | — | |
| GLM-4.7-FlashBase Model=GLM-4.7-Flash2026.05 | 76.5 | — | — | — | |
| ZEDABase Model=GLM-4.7-Flash2026.05 | 73.1 | — | — | — | |
| NETSFT→OPDBase Model=GLM-4.7-Flash2026.05 | 71.8 | — | — | — | |
| NETSFTBase Model=GLM-4.7-Flash2026.05 | 71.4 | — | — | — | |
| ZEDASFTBase Model=GLM-4.7-Flash2026.05 | 71.4 | — | — | — | |
| Qwen3-30B-A3BBase Model=Qwen3-30B-A3B2026.05 | 71 | — | — | — | |
| CoT@5+MajorityModel=OpenAI o3-mini2025.11 | 70 | — | — | — | |
| CoT@5+RLHFlowModel=OpenAI o3-mini2025.11 | 70 | — | — | — | |
| Hermes@1Model=DeepSeek-V3.22025.11 | 70 | — | — | — | |
| Dynamic SkippingBase Model=GLM-4.7-Flash2026.05 | 69.9 | — | — | — | |
| ZEDABase Model=Qwen3-30B-A3B2026.05 | 69.1 | — | — | — | |
| Dynamic SkippingBase Model=Qwen3-30B-A3B2026.05 | 67.9 | — | — | — | |
| NETSFT→OPDBase Model=Qwen3-30B-A3B2026.05 | 67.6 | — | — | — | |
| ZEDASFTBase Model=Qwen3-30B-A3B2026.05 | 66.2 | — | — | — | |
| NETSFTBase Model=Qwen3-30B-A3B2026.05 | 65.7 | — | — | — | |
| CoT@1Model=OpenAI o3-mini2025.11 | 63.3 | — | — | — | |
| CoT@5+MajorityModel=DeepSeek-V3.22025.11 | 60 | — | — | — | |
| CoT@5+ShepherdModel=DeepSeek-V3.22025.11 | 60 | — | — | — | |
| CoT@5+RLHFlowModel=DeepSeek-V3.22025.11 | 60 | — | — | — | |
| CoT@5+Safe*Model=DeepSeek-V3.22025.11 | 60 | — | — | — | |
| CoT@5+SafeModel=DeepSeek-V3.22025.11 | 56.7 | — | — | — | |
| CoT@5+SkyworkModel=DeepSeek-V3.22025.11 | 53.3 | — | — | — | |
| CoT@5+ArmoRMModel=DeepSeek-V3.22025.11 | 53.3 | — | — | — | |
| Qwen3-8B (Thinking)Model=Qwen3-8B, Variant=Thinking2026.05 | 52.5 | — | 73.3 | — | |
| CoT@1Model=DeepSeek-V3.22025.11 | 50 | — | — | — | |
| Weak-to-Strong (AEA)Base Model=Claude 3.7 Sonnet, Aligned=true, AEA Model=AEA-4B2026.05 | 46.6 | — | — | — | |
| Hermes@5+SkyworkModel=Qwen3-8B2025.11 | 43.3 | — | — | — | |
| AdaMoEBase Model=GLM-4.7-Flash2026.05 | 42.4 | — | — | — | |
| Qwen3-1.7B + Guided (ρ¯ ≈ 0.16)Model=Qwen3-1.7B, Intervention Rate (rho)=≈ 0.162026.05 | 36.2 | — | 66.7 | — | |
| MetaAgent-X RLTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 33.33 | — | — | — | |
| Multi AgentBase Model=GPT-4o, Aligned=false, AEA Model=None2026.05 | 33.3 | — | — | — | |
| Weak-to-Strong (AEA)Base Model=GPT-4o, Aligned=true, AEA Model=AEA-4B2026.05 | 33.3 | — | — | — | |
| Hermes@5+MajorityModel=Qwen3-8B2025.11 | 33.3 | — | — | — | |
| Qwen3-1.7B (Thinking)Model=Qwen3-1.7B, Variant=Thinking2026.05 | 32.8 | — | 53.3 | — | |
| Weak-to-Strong (AEA)Base Model=GPT-4o-mini, Aligned=true, AEA Model=AEA-4B2026.05 | 30 | — | — | — | |
| CoT@5+SkyworkModel=Qwen3-8B2025.11 | 30 | — | — | — | |
| CoT@5+ArmoRMModel=Qwen3-8B2025.11 | 30 | — | — | — | |
| Hermes@1Model=Qwen3-8B2025.11 | 30 | — | — | — | |
| MaASTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 29.2 | — | — | — | |
| Qwen3-0.6B + Guided (ρ¯ ≈ 0.13)Model=Qwen3-0.6B, Intervention Rate (rho)=≈ 0.132026.05 | 26.7 | — | 56.7 | — | |
| Self-Reflection (AEA)Base Model=Claude 3.5 Sonnet, Aligned=true, AEA Model=Claude 3.5 Sonnet2026.05 | 26.7 | — | — | — | |
| Weak-to-Strong (AEA)Base Model=Claude 3.5 Sonnet, Aligned=true, AEA Model=AEA-4B2026.05 | 26.7 | — | — | — | |
| Multi AgentBase Model=Claude 3.7 Sonnet, Aligned=false, AEA Model=None2026.05 | 26.7 | — | — | — | |
| Multi AgentBase Model=GPT-4o-mini, Aligned=false, AEA Model=None2026.05 | 26.7 | — | — | — | |
| Self-Reflection (AEA)Base Model=GPT-4o, Aligned=true, AEA Model=GPT-4o2026.05 | 26.7 | — | — | — | |
| SA + GRPOTraining Paradigm=Single Agent, Backbone=Qwen3 8B2026.05 | 26.67 | — | — | — | |
| AdaMoEBase Model=Qwen3-30B-A3B2026.05 | 24.8 | — | — | — | |
| CoT@5+ShepherdModel=Qwen3-8B2025.11 | 23.3 | — | — | — | |
| CoT@5+SafeModel=Qwen3-8B2025.11 | 23.3 | — | — | — | |
| CoT@5+Safe*Model=Qwen3-8B2025.11 | 23.3 | — | — | — | |
| SATraining Paradigm=Single Agent, Backbone=Qwen3 8B2026.05 | 20.9 | — | — | — | |
| AFlowTraining Paradigm=Search-based Auto MAS, Backbone=Qwen3 8B2026.05 | 20.83 | — | — | — | |
| ScoreFlowTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 20 | — | — | — | |
| MetaAgent-X SFTTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 20 | — | — | — | |
| Self-Reflection (AEA)Base Model=GPT-4o-mini, Aligned=true, AEA Model=GPT-4o-mini2026.05 | 20 | — | — | — | |
| CoT@1Model=Qwen3-8B2025.11 | 20 | — | — | — | |
| CoT@5+RLHFlowModel=Qwen3-8B2025.11 | 20 | — | — | — | |
| Qwen3-1.7B + Guided (ρ¯ ≈ 0.06)Model=Qwen3-1.7B, Intervention Rate (rho)=≈ 0.062026.05 | 18.8 | — | 43.3 | — | |
| RandLoRAModel=Qwen3-1.7B2026.05 | 18.3 | — | — | — | |
| IBTPOBackbone=Qwen3-14B-Base2026.05 | 17.7 | — | — | — | |
| GRPO w/ Entropy RegBackbone=Qwen3-14B-Base2026.05 | 17.2 | — | — | — | |
| ADASTraining Paradigm=Search-based Auto MAS, Backbone=Qwen3 8B2026.05 | 16.7 | — | — | — | |
| Weak-to-Strong (AEA)Base Model=Claude 3 Haiku, Aligned=true, AEA Model=AEA-4B2026.05 | 16.7 | — | — | — | |
| Multi AgentBase Model=Claude 3.5 Sonnet, Aligned=false, AEA Model=None2026.05 | 16.7 | — | — | — | |
| Self-Reflection (AEA)Base Model=Claude 3.7 Sonnet, Aligned=true, AEA Model=Claude 3.7 Sonnet2026.05 | 16.7 | — | — | — | |
| CoT@5+MajorityModel=Qwen3-8B2025.11 | 16.7 | — | — | — | |
| Qwen3-0.6B (Thinking)Model=Qwen3-0.6B, Variant=Thinking2026.05 | 16.1 | — | 36.7 | — | |
| IBTPOMaximum truncation length=8K2026.05 | 15.7 | — | — | — | |
| IBROBackbone=Qwen3-14B-Base2026.05 | 15.7 | — | — | — | |
| IBTPOMaximum truncation length=4K2026.05 | 15.4 | — | — | — | |
| TreeRLMaximum truncation length=8K2026.05 | 15.1 | — | — | — | |
| TreeRLMaximum truncation length=4K2026.05 | 15 | — | — | — | |
| DoRAModel=Qwen3-1.7B2026.05 | 14.7 | — | — | — | |
| Full FTModel=Qwen3-1.7B2026.05 | 14.6 | — | — | — | |
| Vanilla GRPOMaximum truncation length=8K2026.05 | 14.3 | — | — | — | |
| Vanilla GRPOMaximum truncation length=4K2026.05 | 13.9 | — | — | — | |
| FURAModel=Qwen3-1.7B2026.05 | 13.8 | — | — | — | |
| Self-Reflection (AEA)Base Model=Claude 3 Haiku, Aligned=true, AEA Model=Claude 3 Haiku2026.05 | 13.3 | — | — | — | |
| Single AgentBase Model=Claude 3.7 Sonnet, Aligned=false, AEA Model=None2026.05 | 13.3 | — | — | — | |
| Single AgentBase Model=GPT-4o-mini, Aligned=false, AEA Model=None2026.05 | 13.3 | — | — | — | |
| TreeRLBackbone=Qwen3-14B-Base2026.05 | 12.5 | — | — | — | |
| Vanilla GRPOBackbone=Qwen3-14B-Base2026.05 | 12.4 | — | — | — | |
| MiLoRAModel=Qwen3-1.7B2026.05 | 11.7 | — | — | — | |
| LoRAModel=Qwen3-1.7B2026.05 | 11.1 | — | — | — | |
| Single AgentBase Model=GPT-4o, Aligned=false, AEA Model=None2026.05 | 10 | — | — | — | |
| Initial ModelBackbone=Qwen3-14B-Base2026.05 | 9.2 | — | — | — | |
| Initial ModelMaximum truncation length=8K2026.05 | 9.1 | — | — | — | |
| Initial ModelMaximum truncation length=4K2026.05 | 8.8 | — | — | — | |
| AFM-CoderTraining Paradigm=RL-based Auto MAS, Backbone=Qwen3 8B2026.05 | 8 | — | — | — | |
| Qwen3-0.6B + Guided (ρ¯ ≈ 0.04)Model=Qwen3-0.6B, Intervention Rate (rho)=≈ 0.042026.05 | 7.5 | — | 20 | — |