Mathematical Reasoning on Olympiad Bench (Accuracy, Generation Length)
85.2AccuracyBCR-Qwen3-4B (Ours)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BCR-Qwen3-4B (Ours)Inference mode=N=12026.04 | 85.2 | 10,717 | |
| Qwen3-4B-Thinking-2507Inference mode=N=12026.04 | 83.3 | 13,069 | |
| Critique-GRPOCritique Model=GPT-4o, Decoding budget=16,384 tokens2025.06 | 69.3 | — | |
| Qwen3-1.7BInference mode=N=12026.04 | 67.3 | 9,395 | |
| e3-1.7BInference mode=N=1, Length Control=true2026.04 | 67.1 | 7,515 | |
| R1-GRPOCritique Model=None, Decoding budget=16,384 tokens2025.06 | 66.4 | — | |
| BCR-JustRL-1.5B (Ours)Inference mode=N=12026.04 | 62.9 | 2,969 | |
| Qwen3-32BCritique Model=None, Decoding budget=16,384 tokens2025.06 | 62.7 | — | |
| JustRL-deepseek-1.5BInference mode=N=12026.04 | 62.1 | 7,017 | |
| BroRL-1.5BInference mode=N=1, Length Control=true2026.04 | 61.5 | 4,352 | |
| DDPOBackbone=Qwen3-8B2026.03 | 58.95 | 5,501 | |
| GRPOBackbone=Qwen3-8B2026.03 | 57.92 | 7,372 | |
| L1Backbone=Qwen3-14B2026.03 | 57.67 | 6,262 | |
| DDPOBackbone=Qwen3-4B2026.03 | 57.27 | 5,721 | |
| GRPOBackbone=Qwen3-4B2026.03 | 56.41 | 7,347 | |
| DDPOBackbone=Qwen3-14B2026.03 | 55.7 | 5,299 | |
| GRPOBackbone=Qwen3-14B2026.03 | 55.39 | 5,885 | |
| STILL-3-1.5BInference mode=N=12026.04 | 55.2 | 9,755 | |
| SHORTER BETTERBackbone=Qwen3-14B2026.03 | 54.23 | 4,920 | |
| A-DLPBackbone=Qwen3-14B2026.03 | 54.03 | 4,419 | |
| GRPO-LEADBackbone=Qwen3-14B2026.03 | 53.64 | 6,173 | |
| DeepSeek-R1-Distill-Qwen-14BModel Scale=14B, Merging Strategy=Distilled Model2026.04 | 53.5 | 5,765.1 | |
| DeepSeek-R1-Distill-Qwen-1.5BInference mode=N=12026.04 | 53.4 | 11,599 | |
| Evo-L2S (Accuracy)Model Scale=14B, Merging Strategy=Single-Objective Evolutionary Merging2026.04 | 51.7 | 5,406.2 | |
| Evo-L2S-02Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 49.9 | 5,051.1 | |
| Evo-L2S-01Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 49.8 | 5,318.2 | |
| Task ArithmeticModel Scale=14B, Merging Strategy=Arithmetic Merging2026.04 | 49.6 | 5,190.8 | |
| Evo-L2S-04Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 47 | 1,885.5 | |
| ACM-TIESModel Scale=14B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 46.5 | 2,145.8 | |
| Evo-L2S-03Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 45.9 | 4,871 | |
| ACM-AverageModel Scale=14B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 45.6 | 2,218 | |
| TIES-MergingModel Scale=14B, Merging Strategy=Arithmetic Merging2026.04 | 44.9 | 3,140.4 | |
| Average MergingModel Scale=14B, Merging Strategy=Arithmetic Merging2026.04 | 44.7 | 3,227.1 | |
| CoDMethod Category=Prompt-guided2026.01 | 41.6 | 6,164 | |
| Ada-R1Method Category=Training-based2026.01 | 40.9 | 2,398 | |
| Evo-L2S-01Model Scale=1.5B, Merging Strategy=Multi-Objective Evolutionary Merging, Pareto Configuration Index=012026.04 | 40.9 | 2,349.8 | |
| AIMMethod Category=Data-dependent Merging2026.01 | 40 | 2,068 | |
| RPAMMethod Category=Data-dependent Merging2026.01 | 39.9 | 2,915 | |
| ACMMethod Category=Data-dependent Merging2026.01 | 39.4 | 3,452 | |
| Evo-L2S-02Model Scale=1.5B, Merging Strategy=Multi-Objective Evolutionary Merging, Pareto Configuration Index=022026.04 | 39.4 | 2,427.1 | |
| Evo-L2S (Accuracy)Model Scale=1.5B, Merging Strategy=Single-Objective Evolutionary Merging, Optimization Objective=Accuracy2026.04 | 39 | 3,651.2 | |
| Evo-L2S (Length)Model Scale=14B, Merging Strategy=Single-Objective Evolutionary Merging2026.04 | 38.7 | 1,120.2 | |
| Evo-L2S-05Model Scale=14B, Merging Strategy=Multi-Objective Evolutionary Merging2026.04 | 37.3 | 1,053.8 | |
| Average MergingModel Scale=1.5B, Merging Strategy=Arithmetic Merging2026.04 | 36.4 | 3,057.5 | |
| ACM-TAModel Scale=1.5B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 36.3 | 3,140.1 | |
| Average MergingMethod Category=Data-free Merging2026.01 | 36 | 1,659 | |
| TIES MergingMethod Category=Data-free Merging2026.01 | 36 | 1,714 | |
| DeepSeek-R1-Distill-Qwen-1.5BModel Scale=1.5B2026.04 | 35 | 5,814.2 | |
| ACM-TAModel Scale=14B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 34.8 | 7,501.5 | |
| Evo-L2S-03Model Scale=1.5B, Merging Strategy=Multi-Objective Evolutionary Merging, Pareto Configuration Index=032026.04 | 34.8 | 2,038.5 | |
| TIES-MergingModel Scale=1.5B, Merging Strategy=Arithmetic Merging2026.04 | 33.6 | 3,828.8 | |
| ACM-AverageModel Scale=1.5B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 33.5 | 2,539.7 | |
| ACM-TIESModel Scale=1.5B, Merging Strategy=Activation-informed Merging (ACM)2026.04 | 32.7 | 2,698.9 | |
| ARM-3BInference mode=N=1, Adaptive Reasoning=true2026.04 | 31.9 | 4,534 | |
| Task ArithmeticModel Scale=1.5B, Merging Strategy=Arithmetic Merging2026.04 | 31.1 | 5,186.3 | |
| DeepSeek-R1-1.5BReasoning Strategy=Long-CoT2026.01 | 30.4 | 7,389 | |
| Evo-L2S-04Model Scale=1.5B, Merging Strategy=Multi-Objective Evolutionary Merging, Pareto Configuration Index=042026.04 | 29.8 | 1,235.1 | |
| Task ArithmeticMethod Category=Data-free Merging2026.01 | 29.2 | 898 | |
| Evo-L2S (Length)Model Scale=1.5B, Merging Strategy=Single-Objective Evolutionary Merging, Optimization Objective=Length2026.04 | 28.4 | 1,258.4 | |
| DARE-LinearMethod Category=Data-free Merging2026.01 | 27 | 987 | |
| Thinker-Q1.5BInference mode=N=1, Adaptive Reasoning=true2026.04 | 25.2 | 813 | |
| Qwen2.5-Math-1.5BReasoning Strategy=Short-CoT2026.01 | 22.8 | 608 | |
| Qwen2.5-14BModel Scale=14B, Merging Strategy=Base Model2026.04 | 21.3 | 1,310.9 | |
| ThinkPruneBase Model=DeepSeek-1.5B, Constraint=1k2026.02 | 14.44 | 14,326 | |
| ThinkPruneBase Model=DeepSeek-1.5B, Constraint=5002026.02 | 13.63 | 14,791 | |
| CRTBase Model=DeepSeek-1.5B2026.02 | 13.63 | 13,956 | |
| DeepSeek-1.5BModel Type=Base Model2026.02 | 13.25 | 15,527 | |
| O1-PrunerBase Model=DeepSeek-1.5B, Ratio=1.02026.02 | 12.81 | 14,410 | |
| O1-PrunerBase Model=DeepSeek-1.5B, Ratio=4.02026.02 | 12.62 | 14,547 | |
| ACPOBase Model=DeepSeek-1.5B2026.02 | 12.62 | 14,189 | |
| L1Base Model=DeepSeek-1.5B2026.02 | 12.56 | 15,533 | |
| Qwen2.5-Math-1.5BModel Scale=1.5B2026.04 | 11.9 | 1,337.7 | |
| ShorterBetterBase Model=DeepSeek-1.5B2026.02 | 11.62 | 12,908 |