List your company
ResearchFormerly WizWand

Mathematical Reasoning on AIME 24/25 (Accuracy)

80AccuracyQwen3-4B-Thinking
10.21628.33346.4564.567Jan 6, 2026Jan 29, 2026Feb 22, 2026Mar 18, 2026Apr 10, 2026May 4, 2026May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Qwen3-4B-ThinkingSteering Mechanism=Self-Reading Steering (+ CAA)2026.0480
Qwen3-4B-ThinkingSteering Mechanism=CAA2026.0478.3
Qwen3-4B-ThinkingSteering Mechanism=Base2026.0476.7
R1-Distill-Qwen-7BSteering Mechanism=Self-Reading Steering (+ CAA)2026.0451.7
R1-Distill-Qwen-7BSteering Mechanism=CAA2026.0448.3
R1-Distill-Llama-8BSteering Mechanism=Self-Reading Steering (+ CAA)2026.0448.3
R1-Distill-Qwen-7BSteering Mechanism=Base2026.0446.7
R1-Distill-Llama-8BSteering Mechanism=CAA2026.0445
DECENTMEMBackbone=GEMMA4-E4B, Framework=AgentNet2026.0543.33
R1-Distill-Llama-8BSteering Mechanism=Base2026.0441.7
DECENTMEMBackbone=GEMMA4-E4B, Framework=DyLAN2026.0541.67
ChatDevBackbone=GEMMA4-E4B, Framework=AutoGen2026.0541.67
DECENTMEMBackbone=QWEN3-14B, Framework=AgentNet2026.0541.66
DECENTMEMBackbone=QWEN3-8B, Framework=DyLAN2026.0540
DECENTMEMBackbone=QWEN3-14B, Framework=DyLAN2026.0540
DECENTMEMBackbone=GEMMA4-E4B, Framework=AutoGen2026.0540
ChatDevBackbone=GEMMA4-E4B, Framework=AgentNet2026.0538.33
ChatDevBackbone=GEMMA4-E4B, Framework=DyLAN2026.0538.33
G-MemoryBackbone=GEMMA4-E4B, Framework=AutoGen2026.0538.33
DECENTMEMBackbone=QWEN3-8B, Framework=AgentNet2026.0536.67
ChatDevBackbone=QWEN3-14B, Framework=AgentNet2026.0536.67
No memoryBackbone=GEMMA4-E4B, Framework=DyLAN2026.0536.67
G-MemoryBackbone=GEMMA4-E4B, Framework=DyLAN2026.0536.67
MetaGPTBackbone=GEMMA4-E4B, Framework=AutoGen2026.0536.67
MiMo-V2-Flash Base# Shots=2-shot, # Activated Params=15B, # Total Params=309B2026.0135.3
ChatDevBackbone=QWEN3-14B, Framework=DyLAN2026.0535
DECENTMEMBackbone=QWEN3-14B, Framework=AutoGen2026.0535
No memoryBackbone=GEMMA4-E4B, Framework=AutoGen2026.0535
ChatDevBackbone=QWEN3-14B, Framework=AutoGen2026.0533.33
G-MemoryBackbone=QWEN3-14B, Framework=AutoGen2026.0533.33
DECENTMEMBackbone=GEMMA4-E2B, Framework=AutoGen2026.0533.33
No memoryBackbone=GEMMA4-E4B, Framework=AgentNet2026.0533.33
ChatDevBackbone=QWEN3-8B, Framework=AgentNet2026.0531.67
ChatDevBackbone=QWEN3-8B, Framework=DyLAN2026.0531.67
DECENTMEMBackbone=QWEN3-8B, Framework=AutoGen2026.0531.67
G-MemoryBackbone=QWEN3-14B, Framework=DyLAN2026.0531.67
MetaGPTBackbone=GEMMA4-E4B, Framework=DyLAN2026.0531.67
Kimi-K2 Base# Shots=2-shot, # Activated Params=32B, # Total Params=1043B2026.0131.6
G-MemoryBackbone=QWEN3-8B, Framework=DyLAN2026.0530
No memoryBackbone=QWEN3-14B, Framework=DyLAN2026.0530
No memoryBackbone=QWEN3-14B, Framework=AutoGen2026.0530
ChatDevBackbone=GEMMA4-E2B, Framework=AutoGen2026.0530
G-MemoryBackbone=GEMMA4-E4B, Framework=AgentNet2026.0530
No memoryBackbone=QWEN3-8B, Framework=AutoGen2026.0528.33
ChatDevBackbone=QWEN3-8B, Framework=AutoGen2026.0528.33
ChatDevBackbone=GEMMA4-E2B, Framework=DyLAN2026.0528.33
DECENTMEMBackbone=GEMMA4-E2B, Framework=DyLAN2026.0528.33
MetaGPTBackbone=GEMMA4-E2B, Framework=AutoGen2026.0528.33
G-MemoryBackbone=GEMMA4-E2B, Framework=AutoGen2026.0528.33
DECENTMEMBackbone=QWEN3-4B, Framework=DyLAN2026.0526.67
No memoryBackbone=QWEN3-8B, Framework=DyLAN2026.0526.67
No memoryBackbone=QWEN3-14B, Framework=AgentNet2026.0526.67
MetaGPTBackbone=QWEN3-14B, Framework=DyLAN2026.0526.67
ChatDevBackbone=GEMMA4-E2B, Framework=AgentNet2026.0526.67
G-MemoryBackbone=QWEN3-8B, Framework=AutoGen2026.0525
DECENTMEMBackbone=GEMMA4-E2B, Framework=AgentNet2026.0525
G-MemoryBackbone=GEMMA4-E2B, Framework=DyLAN2026.0525
No memoryBackbone=GEMMA4-E2B, Framework=AutoGen2026.0525
MetaGPTBackbone=GEMMA4-E4B, Framework=AgentNet2026.0525
DeepSeek-V3.2 Exp Base# Shots=2-shot, # Activated Params=37B, # Total Params=671B2026.0124.8
DECENTMEMBackbone=QWEN3-4B, Framework=AgentNet2026.0523.33
No memoryBackbone=QWEN3-8B, Framework=AgentNet2026.0523.33
MetaGPTBackbone=QWEN3-8B, Framework=DyLAN2026.0523.33
G-MemoryBackbone=QWEN3-14B, Framework=AgentNet2026.0523.33
MetaGPTBackbone=QWEN3-14B, Framework=AutoGen2026.0523.33
No memoryBackbone=GEMMA4-E2B, Framework=AgentNet2026.0523.33
No memoryBackbone=GEMMA4-E2B, Framework=DyLAN2026.0523.33
ChatDevBackbone=QWEN3-4B, Framework=DyLAN2026.0521.67
DECENTMEMBackbone=QWEN3-4B, Framework=AutoGen2026.0521.67
MetaGPTBackbone=GEMMA4-E2B, Framework=DyLAN2026.0521.66
DeepSeek-V3.1 Base# Shots=2-shot, # Activated Params=37B, # Total Params=671B2026.0121.6
ChatDevBackbone=QWEN3-4B, Framework=AutoGen2026.0520
MetaGPTBackbone=QWEN3-8B, Framework=AutoGen2026.0520
MetaGPTBackbone=QWEN3-14B, Framework=AgentNet2026.0520
G-MemoryBackbone=GEMMA4-E2B, Framework=AgentNet2026.0520
BG-MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=20K2026.0218.9
ChatDevBackbone=QWEN3-4B, Framework=AgentNet2026.0518.33
G-MemoryBackbone=QWEN3-4B, Framework=DyLAN2026.0518.33
No memoryBackbone=QWEN3-4B, Framework=AutoGen2026.0518.33
MetaGPTBackbone=GEMMA4-E2B, Framework=AgentNet2026.0518.33
RepeatedFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.0218.3
BG-MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.0218.3
MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=20K2026.0216.7
MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.0216.7
No memoryBackbone=QWEN3-4B, Framework=DyLAN2026.0516.67
G-MemoryBackbone=QWEN3-8B, Framework=AgentNet2026.0516.67
RepeatedFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=20K2026.0216.1
RepeatedFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=10K2026.0215.6
AB-MCTS-MFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.0215.6
MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=10K2026.0215.6
BG-MCTSBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=10K2026.0215.6
AB-MCTS-MFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=20K2026.0215
MetaGPTBackbone=QWEN3-4B, Framework=DyLAN2026.0515
G-MemoryBackbone=QWEN3-4B, Framework=AutoGen2026.0515
SWIFTEvaluation Protocol=Out-of-distribution2026.0414.6
AB-MCTS-MFullBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=10K2026.0214.4
AB-MCTS-MBackbone LLM=Qwen-2.5-7B-Instruct, Token Budget (B)=30K2026.0213.9
MetaGPTBackbone=QWEN3-8B, Framework=AgentNet2026.0513.34
No memoryBackbone=QWEN3-4B, Framework=AgentNet2026.0513.33
DAPOModel Size=14B, RL Algorithm=DAPO2026.0512.9