Lyrics Translation on MAVL English Lyrics ←→ Translated Lyrics 1.0
0.918Syllable Error (SE) - ESSylAVL-CoT (Gemini 2.5)
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SylAVL-CoT (Gemini 2.5)Syllable Constraint (SC)=true, Input Modality=Multimodal, Backbone=Gemini 2.5 Flash2025.05 | 0.918 | 0.909 | 0.66 | 2.507 | 0.059 | 0.086 | 0.046 | 0.151 | 33.4 | 49.8 | 22.5 | 59.6 | |
| SylAVL-CoT (Gemini 2.0)Syllable Constraint (SC)=true, Input Modality=Multimodal, Backbone=Gemini 2.0 Flash2025.05 | 0.966 | 0.902 | 0.695 | 2.572 | 0.063 | 0.089 | 0.049 | 0.155 | 35.2 | 51 | 23.7 | 61.1 | |
| Human ExpertInput Modality=Human2025.05 | 1.78 | 1.031 | 1.052 | 1.716 | 0.115 | 0.098 | 0.141 | 0.096 | 58.7 | 45.5 | 35.3 | 60.7 | |
| GPT-4oSyllable Constraint (SC)=true, Input Modality=Text-only2025.05 | 3.164 | 1.442 | 3.084 | 7.221 | 0.185 | 0.138 | 0.182 | 0.378 | 79.5 | 67.6 | 93.5 | 93.5 | |
| Gemini 2.0Syllable Constraint (SC)=true, Input Modality=Text-only2025.05 | 3.585 | 1.384 | 3.039 | 6.257 | 0.207 | 0.104 | 0.19 | 0.389 | 87.2 | 60.4 | 84.8 | 95.2 | |
| Qwen2.5-72BSyllable Constraint (SC)=true, Input Modality=Text-only2025.05 | 6.512 | 2.729 | 6.502 | 10.061 | 0.354 | 0.183 | 0.361 | 0.517 | 92.7 | 75.4 | 91.1 | 96.2 | |
| Google TranslateSyllable Constraint (SC)=false, Input Modality=Text-only2025.05 | 7.784 | 4.14 | 12.226 | 20.924 | 0.411 | 0.248 | 0.619 | 0.951 | 92.6 | 81.8 | 94.3 | 97.5 | |
| Qwen2.5-72BSyllable Constraint (SC)=false, Input Modality=Text-only2025.05 | 8.063 | 4.132 | 9.87 | 15.654 | 0.426 | 0.249 | 0.515 | 0.75 | 93.9 | 81.6 | 93.9 | 97.3 | |
| GPT-4oSyllable Constraint (SC)=false, Input Modality=Text-only2025.05 | 8.1 | 4.158 | 10.302 | 16.711 | 0.42 | 0.245 | 0.524 | 0.78 | 95 | 82.4 | 94.7 | 98.4 | |
| Gemini 2.0Syllable Constraint (SC)=false, Input Modality=Text-only2025.05 | 8.718 | 4.66 | 10.819 | 16.919 | 0.456 | 0.276 | 0.557 | 0.798 | 94.8 | 82.9 | 94.4 | 97.7 | |
| mBART-50Syllable Constraint (SC)=false, Input Modality=Text-only2025.05 | 25.79 | 4.397 | 13.036 | 15.617 | 1.443 | 0.268 | 0.65 | 0.741 | 95.3 | 83.1 | 95.4 | 95.6 |