Mathematical Reasoning on Minerva Math (Accuracy, Generation Length)
68.4AccuracyBCR-Qwen3-4B (Ours)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BCR-Qwen3-4B (Ours)Inference mode=N=12026.04 | 68.4 | 3,338 | |
| Qwen3-4B-Thinking-2507Inference mode=N=12026.04 | 66.2 | 4,576 | |
| Qwen3-1.7BInference mode=N=12026.04 | 57.7 | 5,714 | |
| e3-1.7BInference mode=N=1, Length Control=true2026.04 | 54.4 | 5,201 | |
| BroRL-1.5BInference mode=N=1, Length Control=true2026.04 | 49.1 | 4,205 | |
| DeepSeek-R1-Distill-Qwen-1.5BInference mode=N=12026.04 | 48.5 | 7,575 | |
| BCR-JustRL-1.5B (Ours)Inference mode=N=12026.04 | 48.5 | 2,494 | |
| STILL-3-1.5BInference mode=N=12026.04 | 46.7 | 5,678 | |
| DeepSeek-R1-Distill-Qwen-14BModel Scale=14B, Merging Strategy=Distilled Model2026.04 | 45.6 | 3,806.2 | |
| Task ArithmeticModel Scale=14B, Merging Strategy=Arithmetic Merging2026.04 | 44.9 | 2,593.1 | |
| Evo-L2S (Accuracy)Model Scale=14B, Merging Strategy=Single-Objective Evolutionary Merging2026.04 | 43.8 | 3,027.3 | |
| Evo-L2S-01Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 43.8 | 2,742.5 | |
| JustRL-deepseek-1.5BInference mode=N=12026.04 | 43.4 | 5,413 | |
| Evo-L2S-02Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 43.4 | 2,429.9 | |
| ACM-AverageModel Scale=14B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 40.8 | 816.7 | |
| Evo-L2S-03Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 40.4 | 1,957.3 | |
| Evo-L2S-04Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 40.4 | 700.8 | |
| TIES-MergingModel Scale=14B, Merging Strategy=Arithmetic Merging2026.04 | 40.1 | 948.6 | |
| CoDMethod Category=Prompt-guided2026.01 | 39.7 | 3,909 | |
| Average MergingModel Scale=14B, Merging Strategy=Arithmetic Merging2026.04 | 39.3 | 965.9 | |
| ACM-TIESModel Scale=14B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 39 | 742.2 | |
| RPAMMethod Category=Data-dependent Merging2026.01 | 38.2 | 1,504 | |
| Evo-L2S-05Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 37.9 | 815.2 | |
| ACMMethod Category=Data-dependent Merging2026.01 | 37.5 | 1,761 | |
| AIMMethod Category=Data-dependent Merging2026.01 | 36.8 | 1,024 | |
| Evo-L2S (Length)Model Scale=14B, Merging Strategy=Single-Objective Evolutionary Merging2026.04 | 34.6 | 832.7 | |
| TIES MergingMethod Category=Data-free Merging2026.01 | 33.8 | 1,181 | |
| ACM-TAModel Scale=1.5B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 33.5 | 1,522.3 | |
| Evo-L2S-01Model Scale=1.5B, Merging Strategy=Multi-Objective Evolutionary Merging, Pareto Configuration Index=012026.04 | 33.5 | 1,059.5 | |
| Evo-L2S-02Model Scale=1.5B, Merging Strategy=Multi-Objective Evolutionary Merging, Pareto Configuration Index=022026.04 | 33.1 | 1,168.4 | |
| Average MergingMethod Category=Data-free Merging2026.01 | 32.7 | 956 | |
| QuestABackbone=OpenMath-Nemotron-1.5B, Sampling Strategy (@k)=@4, Dynamic Sampling=true, Training Steps=2000, Train Batch Size=128, Rollout N=16, Max Context Length=32k, Token Budget=2.6x10^8k2025.12 | 32.08 | — | |
| TIES-MergingModel Scale=1.5B, Merging Strategy=Arithmetic Merging2026.04 | 31.6 | 1,733.2 | |
| ACM-TAModel Scale=14B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 30.9 | 4,903.2 | |
| DeepSeek-R1-1.5BReasoning Strategy=Long-CoT2026.01 | 30.7 | 4,948 | |
| JustRL-NemotronBackbone=OpenMath-Nemotron-1.5B, Sampling Strategy (@k)=@4, Dynamic Sampling=false, Training Steps=3440, Train Batch Size=256, Rollout N=8, Max Context Length=16k, Token Budget=1.1x10^8k2025.12 | 30.24 | — | |
| Average MergingModel Scale=1.5B, Merging Strategy=Arithmetic Merging2026.04 | 30.1 | 1,657.1 | |
| ACM-AverageModel Scale=1.5B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 30.1 | 1,026.5 | |
| Evo-L2S-03Model Scale=1.5B, Merging Strategy=Multi-Objective Evolutionary Merging, Pareto Configuration Index=032026.04 | 30.1 | 1,044.2 | |
| Evo-L2S (Length)Model Scale=1.5B, Merging Strategy=Single-Objective Evolutionary Merging, Optimization Objective=Length2026.04 | 29 | 853.6 | |
| Evo-L2S (Accuracy)Model Scale=1.5B, Merging Strategy=Single-Objective Evolutionary Merging, Optimization Objective=Accuracy2026.04 | 28.3 | 1,673.2 | |
| ARM-3BInference mode=N=1, Adaptive Reasoning=true2026.04 | 27.9 | 3,849 | |
| ACM-TIESModel Scale=1.5B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 27.9 | 1,263.9 | |
| Task ArithmeticMethod Category=Data-free Merging2026.01 | 27.6 | 862 | |
| Evo-L2S-04Model Scale=1.5B, Merging Strategy=Multi-Objective Evolutionary Merging, Pareto Configuration Index=042026.04 | 27.6 | 857.9 | |
| Task ArithmeticModel Scale=1.5B, Merging Strategy=Arithmetic Merging2026.04 | 27.2 | 2,638.1 | |
| BackboneBackbone=OpenMath-Nemotron-1.5B, Sampling Strategy (@k)=@42025.12 | 26.93 | — | |
| Ada-R1Method Category=Training-based2026.01 | 23.5 | 3,021 | |
| Thinker-Q1.5BInference mode=N=1, Adaptive Reasoning=true2026.04 | 23.5 | 777 | |
| DARE-LinearMethod Category=Data-free Merging2026.01 | 22.1 | 872 | |
| DeepSeek-R1-Distill-Qwen-1.5BModel Scale=1.5B2026.04 | 20.2 | 3,014.8 | |
| Qwen2.5-14BModel Scale=14B, Merging Strategy=Base Model2026.04 | 16.5 | 1,315.8 | |
| Qwen2.5-Math-1.5BReasoning Strategy=Short-CoT2026.01 | 11.4 | 1,037 | |
| Qwen2.5-Math-1.5BModel Scale=1.5B2026.04 | 11 | 1,032.2 |