List your company
ResearchFormerly WizWand

Mathematical Reasoning on AMC-23 (test)

97.5AccuracyAgent 1
16.937.82558.7579.675Apr 14, 2026Apr 22, 2026Apr 30, 2026May 8, 2026May 16, 2026May 24, 2026Jun 1, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Agent 1Controller Backbone=Qwen3-4B-Base, Agent 1 Model=Qwen3-30B-A3B-Instruct2026.0597.5
Agent 1Controller Backbone=Qwen3-8B-Base, Agent 1 Model=Qwen3-30B-A3B-Instruct2026.0597.5
Router-R1Controller Backbone=Qwen3-8B-Base2026.0595
Iterative Critique-and-Routing ControllerController Backbone=Qwen3-4B-Base2026.0592.5
Router-R1Controller Backbone=Qwen3-4B-Base2026.0590
Iterative Critique-and-Routing ControllerController Backbone=Qwen3-8B-Base2026.0583.3
Controller V2Controller Backbone=Qwen3-4B-Base2026.0577.5
RoBERTa RouterController Backbone=Qwen3-8B-Base2026.0577.5
Controller V1Controller Backbone=Qwen3-8B-Base2026.0575
Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0670
Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0663.3
RouterDCController Backbone=Qwen3-4B-Base2026.0560
Controller V2Controller Backbone=Qwen3-8B-Base2026.0560
RouterDCController Backbone=Qwen3-8B-Base2026.0560
Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.0659.2
Random RouterController Backbone=Qwen3-4B-Base2026.0557.5
Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0656.7
Controller V1Controller Backbone=Qwen3-4B-Base2026.0555
RoBERTa RouterController Backbone=Qwen3-4B-Base2026.0555
Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0652.5
Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0652.5
Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0652.5
CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.0650.8
Q-RMSFT Model=Qwen3-8B Base, BON=@42026.0450
DPO-RMSFT Model=Qwen3-8B Base, BON=@42026.0450
IPVRMSFT Model=Qwen3-8B Base, BON=@162026.0450
Agent 2Controller Backbone=Qwen3-4B-Base, Agent 2 Model=Qwen2.5-7B-Instruct2026.0550
Random RouterController Backbone=Qwen3-8B-Base2026.0550
Implicit PRMSFT Model=Qwen3-8B Base, BON=@42026.0447.5
IPVRMSFT Model=Qwen3-8B Base, BON=@42026.0447.5
IPVRMSFT Model=Qwen3-8B Base, BON=AVG2026.0447.5
Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.0647.5
Q-RMSFT Model=Qwen3-8B Base, BON=AVG2026.0445.8
Q-RMSFT Model=Qwen3-8B Base, BON=@642026.0445
DPO-RMSFT Model=Qwen3-8B Base, BON=AVG2026.0445
IPVRMSFT Model=Qwen3-8B Base, BON=@642026.0445
Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.0645
Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.0645
Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.0645
Implicit PRMSFT Model=Qwen3-8B Base, BON=AVG2026.0444.2
CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.0643.3
PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0643.3
Q-RMSFT Model=Qwen3-8B Base, BON=@162026.0442.5
DPO-RMSFT Model=Qwen3-8B Base, BON=@162026.0442.5
DPO-RMSFT Model=Qwen3-8B Base, BON=@642026.0442.5
Implicit PRMSFT Model=Qwen3-8B Base, BON=@162026.0442.5
Implicit PRMSFT Model=Qwen3-8B Base, BON=@642026.0442.5
CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.0640.8
LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.0640.8
Agent 2Controller Backbone=Qwen3-8B-Base, Agent 2 Model=Ministral-3-8B-Instruct2026.0540
LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.0640
Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.0640
PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0639.2
Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0638.3
Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0637.5
PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0636.7
Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0636.7
Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0636.7
LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.0635.8
Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0635.8
Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0635.8
Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0635
Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0635
Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0634.2
Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0634.2
Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0632.5
Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0630.8
EndoRMSFT Model=Qwen3-8B Base, BON=@162026.0430
Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0628.3
Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0628.3
Q-RMBackbone=Qwen3-0.6B Base, BON=@64, Training=SFT2026.0427.5
IPVRMBackbone=Qwen3-0.6B Base, BON=@16, Training=SFT2026.0427.5
Agent 3Controller Backbone=Qwen3-4B-Base, Agent 3 Model=Qwen2.5-1.5B-Instruct2026.0527.5
Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0627.5
Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0627.5
Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0627.5
Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0627.5
EndoRMSFT Model=Qwen3-8B Base, BON=AVG2026.0426.7
LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.0626.7
Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0625.8
EndoRMSFT Model=Qwen3-8B Base, BON=@42026.0425
EndoRMSFT Model=Qwen3-8B Base, BON=@642026.0425
Q-RMBackbone=Qwen3-0.6B Base, BON=@4, Training=SFT2026.0425
DPO-RMBackbone=Qwen3-0.6B Base, BON=@64, Training=SFT2026.0425
Implicit PRMBackbone=Qwen3-0.6B Base, BON=@4, Training=SFT2026.0425
IPVRMBackbone=Qwen3-0.6B Base, BON=@4, Training=SFT2026.0425
IPVRMBackbone=Qwen3-0.6B Base, BON=AVG, Training=SFT2026.0425
Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0625
Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0625
CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.0625
LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.0624.2
CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.0624.2
Q-RMBackbone=Qwen3-0.6B Base, BON=AVG, Training=SFT2026.0423.3
DPO-RMBackbone=Qwen3-0.6B Base, BON=AVG, Training=SFT2026.0423.3
CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.0623.3
LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.0623.3
DPO-RMBackbone=Qwen3-0.6B Base, BON=@4, Training=SFT2026.0422.5
DPO-RMBackbone=Qwen3-0.6B Base, BON=@16, Training=SFT2026.0422.5
IPVRMBackbone=Qwen3-0.6B Base, BON=@64, Training=SFT2026.0422.5
EndoRMBackbone=Qwen3-0.6B Base, BON=@64, Training=SFT2026.0420