Machine Translation on English-to-Southeast-Asian Translation Suite LMIC Community Dialogue
47.15BLEU-4 (bn)SAGE
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SAGEBase Model=Qwen-3-8B2026.03 | 47.15 | 48.55 | 48.8 | 41.5 | 37.1 | 33.15 | 48.95 | 86.3 | 86.75 | 86.9 | 84.55 | 83.9 | 82.15 | 86.95 | 60.5 | |
| SAGEBase Model=Llama-3.1-8B2026.03 | 46.9 | 48.2 | 48.55 | 40.8 | 36.55 | 32.5 | 48.6 | 86.1 | 86.5 | 86.75 | 84.2 | 83.65 | 81.9 | 86.7 | 76.2 | |
| SAGEBase Model=Gemma-3-9B2026.03 | 46.75 | 47.9 | 48.3 | 41.15 | 36.88 | 33.25 | 48.45 | 85.95 | 86.4 | 86.6 | 84.05 | 83.5 | 82.2 | 86.65 | 80.1 | |
| DeepSeek-v3Model Category=Open-Source Models2026.03 | 41.85 | 46.5 | 44.1 | 36.95 | 33.1 | 33.5 | 46.8 | 84.55 | 85.05 | 85.15 | 83.1 | 81.85 | 81.6 | 86.05 | 75.6 | |
| Grok-3Model Category=Closed-Source Models2026.03 | 41.5 | 46.1 | 43.85 | 36.2 | 33.55 | 32.9 | 46.15 | 84.1 | 84.9 | 84.95 | 82.88 | 81.5 | 81.75 | 86.1 | 93.4 | |
| Claude-3.5 SonnetModel Category=Closed-Source Models2026.03 | 41.25 | 45.15 | 42.18 | 34.95 | 32.12 | 33.85 | 45.24 | 84.1 | 83.95 | 83.99 | 81.75 | 80.14 | 82.45 | 85.79 | 95.1 | |
| GPT-4oModel Category=Closed-Source Models2026.03 | 40.15 | 45.88 | 42.5 | 35.1 | 28.33 | 31.05 | 45.13 | 83.5 | 84.1 | 84.05 | 81.9 | 79.25 | 80.11 | 85.55 | 92.5 | |
| Gemini-2.5 proModel Category=Closed-Source Models2026.03 | 38.2 | 42.55 | 40.13 | 32.8 | 26.57 | 29.88 | 42.01 | 82.15 | 82.5 | 82.88 | 80.5 | 78.89 | 79.5 | 84.6 | 90.2 | |
| Qwen-3-8BModel Category=Open-Source Models2026.03 | 37.5 | 43.85 | 39.55 | 32.15 | 29.4 | 27.5 | 44.88 | 83.45 | 83.9 | 83.8 | 81.5 | 80.25 | 80.1 | 85.65 | 62.1 | |
| Gemma-3-9BModel Category=Open-Source Models2026.03 | 37.1 | 43.1 | 38.9 | 31.85 | 29.13 | 27.05 | 44.25 | 83.3 | 83.65 | 83.6 | 81.33 | 80.1 | 79.92 | 85.3 | 82.3 | |
| Llama-3.1-8BModel Category=Open-Source Models2026.03 | 36.8 | 43.55 | 39.1 | 31.5 | 28.81 | 26.15 | 44.5 | 83.15 | 83.8 | 83.5 | 81.1 | 79.95 | 79.8 | 85.45 | 78.4 | |
| NLLB-200-3.3BModel Category=Open-Source Models2026.03 | 22.4 | 25.55 | 24.18 | 20.15 | 16.13 | 21.15 | 25.8 | 76.5 | 77.85 | 77.01 | 76.1 | 75.83 | 77.05 | 78.81 | 45.2 | |
| M2M-100-1.2BModel Category=Open-Source Models2026.03 | 3.11 | 2.25 | 2.9 | 1.88 | 0.05 | 0.02 | 3.41 | 61.88 | 60.15 | 61.05 | 59.8 | 56.01 | 56.13 | 62.4 | 46.8 |