Mathematical Reasoning on AIME 24/25 (Accuracy)
80AccuracyQwen3-4B-Thinking
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3-4B-ThinkingSteering Mechanism=Self-Reading Steering (+ CAA)2026.04 | 80 | |
| Qwen3-4B-ThinkingSteering Mechanism=CAA2026.04 | 78.3 | |
| Qwen3-4B-ThinkingSteering Mechanism=Base2026.04 | 76.7 | |
| R1-Distill-Qwen-7BSteering Mechanism=Self-Reading Steering (+ CAA)2026.04 | 51.7 | |
| R1-Distill-Qwen-7BSteering Mechanism=CAA2026.04 | 48.3 | |
| R1-Distill-Llama-8BSteering Mechanism=Self-Reading Steering (+ CAA)2026.04 | 48.3 | |
| R1-Distill-Qwen-7BSteering Mechanism=Base2026.04 | 46.7 | |
| R1-Distill-Llama-8BSteering Mechanism=CAA2026.04 | 45 | |
| DECENTMEMBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 43.33 | |
| R1-Distill-Llama-8BSteering Mechanism=Base2026.04 | 41.7 | |
| DECENTMEMBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 41.67 | |
| ChatDevBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 41.67 | |
| DECENTMEMBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 41.66 | |
| DECENTMEMBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 40 | |
| DECENTMEMBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 40 | |
| DECENTMEMBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 40 | |
| ChatDevBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 38.33 | |
| ChatDevBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 38.33 | |
| G-MemoryBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 38.33 | |
| DECENTMEMBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 36.67 | |
| ChatDevBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 36.67 | |
| No memoryBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 36.67 | |
| G-MemoryBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 36.67 | |
| MetaGPTBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 36.67 | |
| MiMo-V2-Flash Base# Shots=2-shot, # Activated Params=15B, # Total Params=309B2026.01 | 35.3 | |
| ChatDevBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 35 | |
| DECENTMEMBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 35 | |
| No memoryBackbone=GEMMA4-E4B, Framework=AutoGen2026.05 | 35 | |
| ChatDevBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 33.33 | |
| G-MemoryBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 33.33 | |
| DECENTMEMBackbone=GEMMA4-E2B, Framework=AutoGen2026.05 | 33.33 | |
| No memoryBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 33.33 | |
| ChatDevBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 31.67 | |
| ChatDevBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 31.67 | |
| DECENTMEMBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 31.67 | |
| G-MemoryBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 31.67 | |
| MetaGPTBackbone=GEMMA4-E4B, Framework=DyLAN2026.05 | 31.67 | |
| Kimi-K2 Base# Shots=2-shot, # Activated Params=32B, # Total Params=1043B2026.01 | 31.6 | |
| G-MemoryBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 30 | |
| No memoryBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 30 | |
| No memoryBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 30 | |
| ChatDevBackbone=GEMMA4-E2B, Framework=AutoGen2026.05 | 30 | |
| G-MemoryBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 30 | |
| No memoryBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 28.33 | |
| ChatDevBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 28.33 | |
| ChatDevBackbone=GEMMA4-E2B, Framework=DyLAN2026.05 | 28.33 | |
| DECENTMEMBackbone=GEMMA4-E2B, Framework=DyLAN2026.05 | 28.33 | |
| MetaGPTBackbone=GEMMA4-E2B, Framework=AutoGen2026.05 | 28.33 | |
| G-MemoryBackbone=GEMMA4-E2B, Framework=AutoGen2026.05 | 28.33 | |
| DECENTMEMBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 26.67 | |
| No memoryBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 26.67 | |
| No memoryBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 26.67 | |
| MetaGPTBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 26.67 | |
| ChatDevBackbone=GEMMA4-E2B, Framework=AgentNet2026.05 | 26.67 | |
| G-MemoryBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 25 | |
| DECENTMEMBackbone=GEMMA4-E2B, Framework=AgentNet2026.05 | 25 | |
| G-MemoryBackbone=GEMMA4-E2B, Framework=DyLAN2026.05 | 25 | |
| No memoryBackbone=GEMMA4-E2B, Framework=AutoGen2026.05 | 25 | |
| MetaGPTBackbone=GEMMA4-E4B, Framework=AgentNet2026.05 | 25 | |
| DeepSeek-V3.2 Exp Base# Shots=2-shot, # Activated Params=37B, # Total Params=671B2026.01 | 24.8 | |
| DECENTMEMBackbone=QWEN3-4B, Framework=AgentNet2026.05 | 23.33 | |
| No memoryBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 23.33 | |
| MetaGPTBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 23.33 | |
| G-MemoryBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 23.33 | |
| MetaGPTBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 23.33 | |
| No memoryBackbone=GEMMA4-E2B, Framework=AgentNet2026.05 | 23.33 | |
| No memoryBackbone=GEMMA4-E2B, Framework=DyLAN2026.05 | 23.33 | |
| ChatDevBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 21.67 | |
| DECENTMEMBackbone=QWEN3-4B, Framework=AutoGen2026.05 | 21.67 | |
| MetaGPTBackbone=GEMMA4-E2B, Framework=DyLAN2026.05 | 21.66 | |
| DeepSeek-V3.1 Base# Shots=2-shot, # Activated Params=37B, # Total Params=671B2026.01 | 21.6 | |
| ChatDevBackbone=QWEN3-4B, Framework=AutoGen2026.05 | 20 | |
| MetaGPTBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 20 | |
| MetaGPTBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 20 | |
| G-MemoryBackbone=GEMMA4-E2B, Framework=AgentNet2026.05 | 20 | |
| BG-MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=20K2026.02 | 18.9 | |
| ChatDevBackbone=QWEN3-4B, Framework=AgentNet2026.05 | 18.33 | |
| G-MemoryBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 18.33 | |
| No memoryBackbone=QWEN3-4B, Framework=AutoGen2026.05 | 18.33 | |
| MetaGPTBackbone=GEMMA4-E2B, Framework=AgentNet2026.05 | 18.33 | |
| RepeatedFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.02 | 18.3 | |
| BG-MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.02 | 18.3 | |
| MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=20K2026.02 | 16.7 | |
| MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.02 | 16.7 | |
| No memoryBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 16.67 | |
| G-MemoryBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 16.67 | |
| RepeatedFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=20K2026.02 | 16.1 | |
| RepeatedFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=10K2026.02 | 15.6 | |
| AB-MCTS-MFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.02 | 15.6 | |
| MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=10K2026.02 | 15.6 | |
| BG-MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=10K2026.02 | 15.6 | |
| AB-MCTS-MFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=20K2026.02 | 15 | |
| MetaGPTBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 15 | |
| G-MemoryBackbone=QWEN3-4B, Framework=AutoGen2026.05 | 15 | |
| SWIFTEvaluation Protocol=Out-of-distribution2026.04 | 14.6 | |
| AB-MCTS-MFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=10K2026.02 | 14.4 | |
| AB-MCTS-MBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.02 | 13.9 | |
| MetaGPTBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 13.34 | |
| No memoryBackbone=QWEN3-4B, Framework=AgentNet2026.05 | 13.33 | |
| DAPOModel Size=14B, RL Algorithm=DAPO2026.05 | 12.9 |