Mathematical Reasoning on AMC-23 (test)
97.5AccuracyAgent 1
Evaluation Results
| Method | Links | |
|---|---|---|
| Agent 1Controller Backbone=Qwen3-4B-Base, Agent 1 Model=Qwen3-30B-A3B-Instruct2026.05 | 97.5 | |
| Agent 1Controller Backbone=Qwen3-8B-Base, Agent 1 Model=Qwen3-30B-A3B-Instruct2026.05 | 97.5 | |
| Router-R1Controller Backbone=Qwen3-8B-Base2026.05 | 95 | |
| Iterative Critique-and-Routing ControllerController Backbone=Qwen3-4B-Base2026.05 | 92.5 | |
| Router-R1Controller Backbone=Qwen3-4B-Base2026.05 | 90 | |
| Iterative Critique-and-Routing ControllerController Backbone=Qwen3-8B-Base2026.05 | 83.3 | |
| Controller V2Controller Backbone=Qwen3-4B-Base2026.05 | 77.5 | |
| RoBERTa RouterController Backbone=Qwen3-8B-Base2026.05 | 77.5 | |
| Controller V1Controller Backbone=Qwen3-8B-Base2026.05 | 75 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 70 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 63.3 | |
| RouterDCController Backbone=Qwen3-4B-Base2026.05 | 60 | |
| Controller V2Controller Backbone=Qwen3-8B-Base2026.05 | 60 | |
| RouterDCController Backbone=Qwen3-8B-Base2026.05 | 60 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 59.2 | |
| Random RouterController Backbone=Qwen3-4B-Base2026.05 | 57.5 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 56.7 | |
| Controller V1Controller Backbone=Qwen3-4B-Base2026.05 | 55 | |
| RoBERTa RouterController Backbone=Qwen3-4B-Base2026.05 | 55 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 52.5 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 52.5 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 52.5 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 50.8 | |
| Q-RMSFT Model=Qwen3-8B Base, BON=@42026.04 | 50 | |
| DPO-RMSFT Model=Qwen3-8B Base, BON=@42026.04 | 50 | |
| IPVRMSFT Model=Qwen3-8B Base, BON=@162026.04 | 50 | |
| Agent 2Controller Backbone=Qwen3-4B-Base, Agent 2 Model=Qwen2.5-7B-Instruct2026.05 | 50 | |
| Random RouterController Backbone=Qwen3-8B-Base2026.05 | 50 | |
| Implicit PRMSFT Model=Qwen3-8B Base, BON=@42026.04 | 47.5 | |
| IPVRMSFT Model=Qwen3-8B Base, BON=@42026.04 | 47.5 | |
| IPVRMSFT Model=Qwen3-8B Base, BON=AVG2026.04 | 47.5 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 47.5 | |
| Q-RMSFT Model=Qwen3-8B Base, BON=AVG2026.04 | 45.8 | |
| Q-RMSFT Model=Qwen3-8B Base, BON=@642026.04 | 45 | |
| DPO-RMSFT Model=Qwen3-8B Base, BON=AVG2026.04 | 45 | |
| IPVRMSFT Model=Qwen3-8B Base, BON=@642026.04 | 45 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 45 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 45 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 45 | |
| Implicit PRMSFT Model=Qwen3-8B Base, BON=AVG2026.04 | 44.2 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 43.3 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 43.3 | |
| Q-RMSFT Model=Qwen3-8B Base, BON=@162026.04 | 42.5 | |
| DPO-RMSFT Model=Qwen3-8B Base, BON=@162026.04 | 42.5 | |
| DPO-RMSFT Model=Qwen3-8B Base, BON=@642026.04 | 42.5 | |
| Implicit PRMSFT Model=Qwen3-8B Base, BON=@162026.04 | 42.5 | |
| Implicit PRMSFT Model=Qwen3-8B Base, BON=@642026.04 | 42.5 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 40.8 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 40.8 | |
| Agent 2Controller Backbone=Qwen3-8B-Base, Agent 2 Model=Ministral-3-8B-Instruct2026.05 | 40 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 40 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 40 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 39.2 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 38.3 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 37.5 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 36.7 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 36.7 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 36.7 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 35.8 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 35.8 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 35.8 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 35 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 35 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 34.2 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 34.2 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 32.5 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 30.8 | |
| EndoRMSFT Model=Qwen3-8B Base, BON=@162026.04 | 30 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 28.3 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 28.3 | |
| Q-RMBackbone=Qwen3-0.6B Base, BON=@64, Training=SFT2026.04 | 27.5 | |
| IPVRMBackbone=Qwen3-0.6B Base, BON=@16, Training=SFT2026.04 | 27.5 | |
| Agent 3Controller Backbone=Qwen3-4B-Base, Agent 3 Model=Qwen2.5-1.5B-Instruct2026.05 | 27.5 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 27.5 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 27.5 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 27.5 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 27.5 | |
| EndoRMSFT Model=Qwen3-8B Base, BON=AVG2026.04 | 26.7 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 26.7 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 25.8 | |
| EndoRMSFT Model=Qwen3-8B Base, BON=@42026.04 | 25 | |
| EndoRMSFT Model=Qwen3-8B Base, BON=@642026.04 | 25 | |
| Q-RMBackbone=Qwen3-0.6B Base, BON=@4, Training=SFT2026.04 | 25 | |
| DPO-RMBackbone=Qwen3-0.6B Base, BON=@64, Training=SFT2026.04 | 25 | |
| Implicit PRMBackbone=Qwen3-0.6B Base, BON=@4, Training=SFT2026.04 | 25 | |
| IPVRMBackbone=Qwen3-0.6B Base, BON=@4, Training=SFT2026.04 | 25 | |
| IPVRMBackbone=Qwen3-0.6B Base, BON=AVG, Training=SFT2026.04 | 25 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 25 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 25 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 25 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 24.2 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 24.2 | |
| Q-RMBackbone=Qwen3-0.6B Base, BON=AVG, Training=SFT2026.04 | 23.3 | |
| DPO-RMBackbone=Qwen3-0.6B Base, BON=AVG, Training=SFT2026.04 | 23.3 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 23.3 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 23.3 | |
| DPO-RMBackbone=Qwen3-0.6B Base, BON=@4, Training=SFT2026.04 | 22.5 | |
| DPO-RMBackbone=Qwen3-0.6B Base, BON=@16, Training=SFT2026.04 | 22.5 | |
| IPVRMBackbone=Qwen3-0.6B Base, BON=@64, Training=SFT2026.04 | 22.5 | |
| EndoRMBackbone=Qwen3-0.6B Base, BON=@64, Training=SFT2026.04 | 20 |