Mathematical Reasoning on Minerva (Pass@16)
40.9Avg@16Dr. MAS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Dr. MASBackbone=Qwen3-4B, Agent Setting=Multi-Agent & LLM Sharing2026.02 | 40.9 | 53.3 | |
| Dr. MASBackbone=Qwen3-8B, Agent Setting=Multi-Agent & LLM Non-Sharing2026.02 | 40.9 | 54 | |
| Dr. MASBackbone=Qwen3-8B, Agent Setting=Multi-Agent & LLM Sharing2026.02 | 39.9 | 49.6 | |
| GRPOBackbone=Qwen3-8B, Agent Setting=Multi-Agent & LLM Non-Sharing2026.02 | 39.2 | 50.7 | |
| KPO-unclippedClipping strategy=unclipped, Number of candidate solutions=162026.02 | 39.15 | 50.36 | |
| Dr. MASBackbone=Qwen3-4B, Agent Setting=Multi-Agent & LLM Non-Sharing2026.02 | 39 | 51.5 | |
| GRPONumber of candidate solutions=162026.02 | 38.67 | 50.36 | |
| GMPONumber of candidate solutions=162026.02 | 38.48 | 50.73 | |
| KPO-clippedClipping strategy=clipped, Number of candidate solutions=162026.02 | 38.23 | 48.16 | |
| GRPOBackbone=Qwen3-4B, Agent Setting=Single-Agent2026.02 | 37.9 | 49.6 | |
| GRPOBackbone=Qwen3-4B, Agent Setting=Multi-Agent & LLM Sharing2026.02 | 37.5 | 50 | |
| GRPOBackbone=Qwen3-4B, Agent Setting=Multi-Agent & LLM Non-Sharing2026.02 | 37.5 | 50.7 | |
| GRPOBackbone=Qwen3-8B, Agent Setting=Multi-Agent & LLM Sharing2026.02 | 37.5 | 50 | |
| GSPONumber of candidate solutions=162026.02 | 37.17 | 47.79 | |
| Scaf-GRPOBackbone=Qwen2.5-7B2025.10 | 37 | — | |
| Scaf-GRPOBackbone=Qwen2.5-Math-7B2025.10 | 36.8 | — | |
| Vanilla GRPOBackbone=Qwen2.5-7B2025.10 | 36.7 | — | |
| Qwen2.5-7B + DAPOModel Family=Qwen Family, Strategy=DAPO2026.01 | 36.4 | — | |
| GRPOBackbone=Qwen3-8B, Agent Setting=Single-Agent2026.02 | 36 | 46.7 | |
| SENTModel=Qwen3-14B2025.12 | 35.34 | 45.22 | |
| Oat-ZeroBackbone=Qwen2.5-Math-7B2025.10 | 35.2 | — | |
| GRPOModel=Qwen3-14B2025.12 | 35.02 | 44.12 | |
| Scaf-GRPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 35 | — | |
| Qwen2.5-7B + DAPO-SilhouetteModel Family=Qwen Family, Strategy=DAPO-Silhouette2026.01 | 34.3 | — | |
| Vanilla GRPOBackbone=Qwen2.5-Math-7B2025.10 | 33.4 | — | |
| LUFFYBackbone=Qwen2.5-Math-7B2025.10 | 33 | — | |
| Vanilla GRPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 32.5 | — | |
| SimpleRL-ZeroBackbone=Qwen2.5-Math-7B2025.10 | 31.8 | — | |
| Scaf-GRPOBackbone=Qwen2.5-Math-1.5B2025.10 | 31.4 | — | |
| OctoThinker-8B + DAPO-SilhouetteModel Family=OctoThinker Family, Strategy=DAPO-Silhouette2026.01 | 29.6 | — | |
| Vanilla GRPOBackbone=Qwen2.5-Math-1.5B2025.10 | 28.5 | — | |
| OctoThinker-8B + DAPOModel Family=OctoThinker Family, Strategy=DAPO2026.01 | 27.6 | — | |
| Llama-3.1-8B-Instruct + DAPO-SilhouetteModel Family=Llama Family, Strategy=DAPO-Silhouette2026.01 | 25.4 | — | |
| Llama-3.1-8B-Instruct + DAPOModel Family=Llama Family, Strategy=DAPO2026.01 | 25.1 | — | |
| GRPO w/ Maskbase_model=Qwen2.5-Math-7B2025.12 | 22.04 | 39.71 | |
| SENTbase_model=Qwen2.5-Math-7B2025.12 | 21.76 | 40.07 | |
| Llama-3.1-8B-InstructModel Family=Llama Family2026.01 | 21.7 | — | |
| GRPO w/ Enbase_model=Qwen2.5-Math-7B2025.12 | 21.62 | 38.24 | |
| GRPObase_model=Qwen2.5-Math-7B2025.12 | 21.51 | 39.71 | |
| Qwen2.5-7BModel Family=Qwen Family2026.01 | 19.4 | — | |
| Scaf-GRPOBackbone=Llama-3.2-3B-Instruct2025.10 | 19.1 | — | |
| Vanilla GRPOBackbone=Llama-3.2-3B-Instruct2025.10 | 18.7 | — | |
| OctoThinker-8BModel Family=OctoThinker Family2026.01 | 13.3 | — |