Mathematical Reasoning on MSVAMP Bangla
81.1Accuracy (Original)Qwen 3
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen 3Model Category=Reasoning Models, Parameter Count=8B2026.01 | 81.1 | 66.9 | 14.2 | 71.4 | 1,802 | 3,534 | 3,128 | |
| †DAGGERBackbone=Gemma 3, Parameter Count=12B, Training Stage=GRPO2026.01 | 78.8 | 66.8 | 12 | 69.4 | 237 | 356 | 359 | |
| Qwen 3Model Category=Reasoning Models, Parameter Count=4B2026.01 | 78.2 | 63.1 | 15.1 | 67.1 | 1,767 | 3,522 | 3,140 | |
| †DAGGERBackbone=Gemma 3, Parameter Count=12B, Training Stage=SFT2026.01 | 76.8 | 65.4 | 11.5 | 66.7 | 222 | 324 | 334 | |
| †DAGGERBackbone=Gemma 3, Parameter Count=12B, Training Stage=w/o train.2026.01 | 74.2 | 61.1 | 13.1 | 61.7 | 225 | 299 | 316 | |
| Gemma 3Model Category=LLM (5-shot CoT), Parameter Count=12B, Protocol=5-shot CoT2026.01 | 72.3 | 48.7 | 23.6 | 55.7 | 518 | 612 | 599 | |
| †DAGGERBackbone=Gemma 3, Parameter Count=4B, Training Stage=GRPO2026.01 | 70.3 | 42.9 | 27.4 | 47.3 | 252 | 353 | 349 | |
| †DAGGERBackbone=Gemma 3, Parameter Count=4B, Training Stage=SFT2026.01 | 65 | 42.4 | 22.7 | 44.3 | 231 | 373 | 370 | |
| Gemma 3Model Category=LLM (5-shot CoT), Parameter Count=4B, Protocol=5-shot CoT2026.01 | 60.7 | 28.8 | 31.9 | 36.2 | 587 | 743 | 700 | |
| †DAGGERBackbone=Gemma 3, Parameter Count=4B, Training Stage=w/o train.2026.01 | 57.2 | 31.7 | 25.5 | 33.3 | 235 | 308 | 399 | |
| LLaMA 3Model Category=LLM (5-shot CoT), Parameter Count=8B, Protocol=5-shot CoT2026.01 | 54.7 | 26.9 | 27.8 | 33 | 349 | 399 | 386 | |
| Qwen 2.5Model Category=LLM (5-shot CoT), Parameter Count=7B, Protocol=5-shot CoT2026.01 | 53.9 | 35.8 | 18.1 | 37.1 | 413 | 662 | 613 | |
| Qwen 2.5Model Category=LLM (5-shot CoT), Parameter Count=3B, Protocol=5-shot CoT2026.01 | 53.7 | 13 | 40.7 | 22.6 | 336 | 710 | 636 |