Machine Translation on SIGMORPHON 2023 (test)
0.187BLEU (Git)model-gloss
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| model-glossBase Model=Llama-3 70B2024.10 | 0.187 | 0.1394 | 0.1696 | 0.1428 | 0.1597 | 47.89 | 39.65 | 41.56 | 42.3 | 41.45 | 40.83 | |
| LingoLLM w/ GPT-4Base Model=GPT-42024.10 | 0.143 | — | 0.129 | 0.151 | 0.141 | — | — | — | — | — | — | |
| chain-glossBase Model=Llama-3 70B2024.10 | 0.0571 | 0.0729 | 0.0235 | 0.0163 | 0.0425 | 24.66 | 22.62 | 19.19 | 18.01 | 20.84 | 16.78 | |
| gloss-shotBase Model=Llama-3 70B2024.10 | 0.0496 | 0.058 | 0.0132 | 0.0172 | 0.0341 | 25.87 | 23.08 | 20.24 | 20.95 | 22.5 | 18.21 | |
| few-shotBase Model=Llama-3 70B2024.10 | 0.0471 | 0.0636 | 0.0334 | 0.0146 | 0.0394 | 25.18 | 22.89 | 19.41 | 20.03 | 21.85 | 16.76 | |
| zeroCoTBase Model=Llama-3 70B2024.10 | 0.0284 | 0.0174 | 0.0037 | 0.0032 | 0.0132 | 21.21 | 15.27 | 13.95 | 15.68 | 16.53 | 14.5 | |
| zero-shotBase Model=Llama-3 70B2024.10 | 0.0126 | 0.0146 | 0.0026 | 0.0039 | 0.0088 | 23.9 | 17.71 | 13.76 | 16.84 | 18.05 | 15.21 | |
| NLLB-200Base Model=NLLB-2002024.10 | 0.009 | 0.008 | 0.004 | 0.001 | 0.0055 | 23.65 | 18 | 12.3 | 10.1 | 13.8 | 12.82 |