Lyric Translation on MAVL English Lyrics 1.0 (test)
0.905MPNet Similarity (ES)Google Translate
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Google Translatesyllable constraint=false2025.05 | 0.905 | 0.903 | 0.857 | 0.857 | |
| Qwen2.5-72Bsyllable constraint=false2025.05 | 0.9 | 0.899 | 0.841 | 0.836 | |
| GPT-4osyllable constraint=false2025.05 | 0.899 | 0.897 | 0.849 | 0.842 | |
| Gemini 2.0syllable constraint=false2025.05 | 0.893 | 0.891 | 0.843 | 0.834 | |
| Qwen2.5-72Bsyllable constraint=true2025.05 | 0.883 | 0.878 | 0.823 | 0.81 | |
| GPT-4osyllable constraint=true2025.05 | 0.83 | 0.842 | 0.793 | 0.782 | |
| Gemini 2.0syllable constraint=true2025.05 | 0.781 | 0.79 | 0.754 | 0.733 | |
| mBART-50syllable constraint=false2025.05 | 0.766 | 0.89 | 0.85 | 0.842 | |
| SylAVL-CoTsyllable constraint=true, backbone=Gemini 2.02025.05 | 0.765 | 0.786 | 0.73 | 0.695 | |
| SylAVL-CoTsyllable constraint=true, backbone=Gemini 2.52025.05 | 0.76 | 0.788 | 0.732 | 0.697 | |
| Human Expert2025.05 | 0.639 | 0.613 | 0.575 | 0.537 |