Machine Translation on Virām (test)
14.84BLEUGemma 2 9b
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemma 2 9bPrompting Strategy=3-Shot Prompting, Approach=Restore then Translate2025.12 | 14.84 | 0.7407 | 0.6889 | 49.27 | 45.5 | 0.8926 | 0.7056 | |
| Gemma 2 9bPrompting Strategy=Direct Translation using Sentence Meant (Original), Approach=Direct Translation2025.12 | 13.75 | 0.7256 | 0.6941 | 45.85 | 42.47 | 0.8829 | 0.6896 | |
| Gemma 2 9bPrompting Strategy=Zero-Shot, Approach=Restore then Translate2025.12 | 11.69 | 0.726 | 0.6783 | 45.89 | 41.98 | 0.8783 | 0.6952 | |
| Sarvam 2b v0.5Prompting Strategy=3-Shot Prompting, Approach=Direct Translation2025.12 | 11.01 | 0.7571 | 0.7224 | 49.27 | 44.74 | 0.8803 | 0.7019 | |
| Gemma 2 9bPrompting Strategy=3-Shot Prompting, Approach=Direct Translation2025.12 | 10.99 | 0.7388 | 0.6938 | 48.98 | 44.88 | 0.8883 | 0.7013 | |
| Sarvam 2b v0.5Prompting Strategy=Direct Translation using Sentence Meant (Original), Approach=Direct Translation2025.12 | 10.36 | 0.6866 | 0.6725 | 41.3 | 37.63 | 0.8422 | 0.655 | |
| Sarvam 2b v0.5Prompting Strategy=Zero-Shot, Approach=Restore then Translate2025.12 | 9.97 | 0.6961 | 0.6736 | 42.93 | 38.86 | 0.8228 | 0.6468 | |
| Gemma 2 9bPrompting Strategy=Zero-Shot, Approach=Direct Translation2025.12 | 8.94 | 0.7248 | 0.688 | 45.4 | 41.33 | 0.8559 | 0.6712 | |
| Sarvam 2b v0.5Prompting Strategy=3-Shot Prompting, Approach=Restore then Translate2025.12 | 8.55 | 0.7352 | 0.6982 | 44.48 | 40.02 | 0.8509 | 0.671 | |
| Llama 3.1 8bPrompting Strategy=3-Shot Prompting, Approach=Direct Translation2025.12 | 7.98 | 0.6776 | 0.6272 | 43.21 | 38.91 | 0.8216 | 0.6192 | |
| Llama 3.1 8bPrompting Strategy=Direct Translation using Sentence Meant (Original), Approach=Direct Translation2025.12 | 7.87 | 0.6785 | 0.633 | 39.7 | 35.84 | 0.8404 | 0.6535 | |
| Llama 3.1 8bPrompting Strategy=Zero-Shot, Approach=Direct Translation2025.12 | 6.37 | 0.666 | 0.6192 | 39.89 | 35.76 | 0.823 | 0.6314 | |
| Sarvam 2b v0.5Prompting Strategy=Zero-Shot, Approach=Direct Translation2025.12 | 5.67 | 0.63 | 0.6392 | 37.47 | 33.67 | 0.8102 | 0.6149 | |
| Llama 3.1 8bPrompting Strategy=Zero-Shot, Approach=Restore then Translate2025.12 | 5.16 | 0.6548 | 0.6026 | 37.13 | 33.18 | 0.803 | 0.6093 | |
| Llama 3.1 8bPrompting Strategy=3-Shot Prompting, Approach=Restore then Translate2025.12 | 4.56 | 0.6641 | 0.6113 | 37.8 | 33.68 | 0.8194 | 0.612 |