Machine Translation on GlossLM (test)
23.53BLEU (Swahili)chain-gloss
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| chain-glossBackbone=Llama-3 70B, Evaluation Protocol=chain-gloss2024.10 | 23.53 | 14.1 | 5.05 | 17.32 | 25.25 | 17.06 | 45.44 | 33.54 | 24.9 | 35.37 | 46.27 | 37.1 | 35.77 | |
| GRAMMAMTBackbone=Llama-3 70B, Evaluation Protocol=few-shot2024.10 | 22.41 | 11.98 | 6.43 | 19.19 | 23.5 | 16.69 | 45.75 | 29.92 | 28.87 | 36.11 | 44.16 | 36.96 | 34.52 | |
| gloss-shotBackbone=Llama-3 70B, Evaluation Protocol=gloss-shot2024.10 | 22.18 | 16.32 | 3.5 | 17.53 | 22.35 | 16.39 | 46.5 | 33.24 | 25.79 | 36.18 | 42.68 | 36.88 | 35.65 | |
| GRAMMAMTBackbone=Llama-3 70B, Evaluation Protocol=zero-shot2024.10 | 16.99 | 4.48 | 4.92 | 0.7 | 5.84 | 6.58 | 40.35 | 18.87 | 27.97 | 13.28 | 25.65 | 25.22 | 27.1 | |
| GRAMMAMTBackbone=Llama-3 70B, Evaluation Protocol=zero-CoT2024.10 | 15.78 | 1.93 | 4.64 | 1.08 | 4.99 | 5.69 | 39.15 | 18.84 | 28.02 | 14.87 | 25.2 | 25.22 | 27.76 | |
| NLLB-2002024.10 | 6.9 | 0.5 | 3.5 | 0.3 | 0.8 | 2.4 | 24.2 | 10.8 | 21.1 | 10 | 10.7 | 15.36 | 20.21 |