Mathematics on GSM8K (Accuracy)
94.4AccuracyCOSE
Evaluation Results
| Method | Links | |
|---|---|---|
| COSEBackbone=Qwen3-4B2026.05 | 94.4 | |
| AZRBackbone=Qwen3-4B2026.05 | 93.2 | |
| MAEBackbone=Qwen3-4B2026.05 | 92.8 | |
| Fine-tunedBase LLM=Qwen-2.5-7B-Instruct, Parameters=x4, Evaluation Protocol=0 shot, CoT2026.05 | 92.6 | |
| R-ZeroBackbone=Qwen3-4B2026.05 | 92.4 | |
| Zero-shotBase LLM=Qwen-2.5-7B-Instruct, Parameters=x1, Evaluation Protocol=0 shot, CoT2026.05 | 91.7 | |
| DiDi-Merg.-LBase LLM=Qwen-2.5-7B-Instruct, Parameters=x2.0, Evaluation Protocol=0 shot, CoT2026.05 | 91.6 | |
| Twin-MergingBase LLM=Qwen-2.5-7B-Instruct, Parameters=x2.25, Evaluation Protocol=0 shot, CoT2026.05 | 91.3 | |
| FREE-MergingBase LLM=Qwen-2.5-7B-Instruct, Parameters=x2.08, Evaluation Protocol=0 shot, CoT2026.05 | 91.3 | |
| COSEBackbone=Llama-3.2-3B-Instruct2026.05 | 89 | |
| Fine-tunedBase LLM=Llama-3.1-8B-Instruct, Parameters=x4, Evaluation Protocol=0 shot, CoT2026.05 | 88.8 | |
| FREE-MergingBase LLM=Llama-3.1-8B-Instruct, Parameters=x2.08, Evaluation Protocol=0 shot, CoT2026.05 | 88.2 | |
| DiDi-Merg.-LBase LLM=Llama-3.1-8B-Instruct, Parameters=x2.0, Evaluation Protocol=0 shot, CoT2026.05 | 88 | |
| Twin-MergingBase LLM=Llama-3.1-8B-Instruct, Parameters=x2.25, Evaluation Protocol=0 shot, CoT2026.05 | 86.4 | |
| Zero-shotBase LLM=Llama-3.1-8B-Instruct, Parameters=x1, Evaluation Protocol=0 shot, CoT2026.05 | 85.9 | |
| BaseBackbone=Qwen3-4B2026.05 | 85 | |
| MAEBackbone=Llama-3.2-3B-Instruct2026.05 | 75.2 | |
| R-ZeroBackbone=Llama-3.2-3B-Instruct2026.05 | 73.8 | |
| AZRBackbone=Llama-3.2-3B-Instruct2026.05 | 72.4 | |
| BaseBackbone=Llama-3.2-3B-Instruct2026.05 | 63 | |
| pmathModel variant=Mathematics specialized expert2026.05 | 50.2 | |
| Model Swarms2026.06 | 46.85 | |
| MERGEvolve2026.06 | 42.5 | |
| LoraHub2026.06 | 41.8 | |
| EMM2026.06 | 41.25 | |
| p^ (selective composition)Model variant=Composed model (pbase, pmath, pcoding)2026.05 | 41.2 | |
| Expert Fusion2026.06 | 40.52 | |
| Pack of LLMs2026.06 | 37.9 | |
| pbaseModel variant=Gemma 2 2B base model2026.05 | 3.2 | |
| pcodingModel variant=Coding specialized expert2026.05 | 3.2 |