Logical Reasoning on LRLs Bengali Swahili Telugu
79BPAGemma-3-12B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemma-3-12BConfiguration=Steering2026.02 | 79 | — | |
| Gemma-2-9BConfiguration=SFT2026.02 | 77 | 3.5 | |
| Gemma-2-9BConfiguration=Steering2026.02 | 73.5 | — | |
| Qwen-3-14BConfiguration=Steering2026.02 | 70.8 | — | |
| Qwen-2.5-7BConfiguration=SFT2026.02 | 70.1 | 5.3 | |
| Ministral-14BConfiguration=Steering2026.02 | 69.2 | — | |
| Qwen-2.5-7BConfiguration=Steering2026.02 | 64.8 | — | |
| Gemma-3-12BConfiguration=Base Model2026.02 | 58 | — | |
| Qwen-2.5-7BConfiguration=Base Model2026.02 | 55.2 | — | |
| Qwen-2.5-7BConfiguration=CoT2026.02 | 55 | -9.8 | |
| Qwen-3-14BConfiguration=Base Model2026.02 | 54 | — | |
| Mistral-7BConfiguration=SFT2026.02 | 53.7 | 0.7 | |
| Gemma-2-9BConfiguration=CoT2026.02 | 53.6 | -19.9 | |
| Mistral-7BConfiguration=Steering2026.02 | 53 | — | |
| Ministral-14BConfiguration=Base Model2026.02 | 50.4 | — | |
| Mistral-7BConfiguration=CoT2026.02 | 50 | -3 | |
| Mistral-7BConfiguration=Base Model2026.02 | 47.6 | — | |
| Gemma-2-9BConfiguration=Base Model2026.02 | 47 | — |