Lyrics Translation Semantic Evaluation on English Lyrics ↔ Translated Lyrics
22.9MiniLM Score (ES)Google Translate
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Google TranslateSelf-Consistency=false2025.05 | 22.9 | 23.22 | 9.12 | 8.1 | 90.52 | 90.27 | 85.72 | 85.69 | 77.51 | 78.11 | 68.05 | 67.43 | |
| Qwen-72BSelf-Consistency=false2025.05 | 21.74 | 22.58 | 6.57 | 7.26 | 90.04 | 89.87 | 84.12 | 83.62 | 77.25 | 77.71 | 68.49 | 67.84 | |
| GPT-4oSelf-Consistency=false2025.05 | 21.31 | 21.67 | 6.28 | 7.06 | 89.93 | 89.69 | 84.86 | 84.22 | 76.88 | 77.27 | 68.12 | 67.47 | |
| Gemini 2.0Self-Consistency=false2025.05 | 20.61 | 21.46 | 6.87 | 7.25 | 89.31 | 89.11 | 84.26 | 83.37 | 75.75 | 76.24 | 67.47 | 66.21 | |
| Qwen-72BSelf-Consistency=true2025.05 | 20.42 | 20.46 | 6.53 | 7.78 | 88.25 | 87.76 | 82.26 | 81.03 | 76.15 | 76.56 | 68.12 | 67.64 | |
| GPT-4oSelf-Consistency=true2025.05 | 20.42 | 20.47 | 7.03 | 8.37 | 82.95 | 84.17 | 79.32 | 78.18 | 74.21 | 75.42 | 67.59 | 67.11 | |
| MBart-50Self-Consistency=false2025.05 | 19.93 | 24.91 | 8.56 | 9.94 | 76.58 | 88.95 | 84.96 | 84.15 | 73.31 | 77.75 | 66.77 | 68.28 | |
| SylAVL-CoT (Gemini 2.0)Self-Consistency=true2025.05 | 19 | 18.49 | 6.79 | 8.67 | 76.52 | 78.59 | 72.89 | 69.53 | 71.42 | 72.78 | 66.35 | 65.67 | |
| Gemini 2.0Self-Consistency=true2025.05 | 18.57 | 17.85 | 6.73 | 8.26 | 78.13 | 79.04 | 75.44 | 73.27 | 71.69 | 72.34 | 66.86 | 66.04 | |
| Human Expert2025.05 | 18.04 | 16.22 | 7.55 | 8.01 | 63.92 | 61.26 | 57.46 | 53.74 | 68.56 | 68.67 | 64.86 | 63.83 |