Lyrics Translation Semantic Evaluation on English Lyrics ↔ Translated Lyrics benchmark leaderboard · SOTA2 Research

Lyrics Translation Semantic Evaluation on English Lyrics ↔ Translated Lyrics

22.9MiniLM Score (ES)Google Translate

Evaluation Results

Method	Links
Google Translate2025.05		22.9	23.22	9.12	8.1	90.52	90.27	85.72	85.69	77.51	78.11	68.05	67.43
Qwen-72B2025.05		21.74	22.58	6.57	7.26	90.04	89.87	84.12	83.62	77.25	77.71	68.49	67.84
GPT-4o2025.05		21.31	21.67	6.28	7.06	89.93	89.69	84.86	84.22	76.88	77.27	68.12	67.47
Gemini 2.02025.05		20.61	21.46	6.87	7.25	89.31	89.11	84.26	83.37	75.75	76.24	67.47	66.21
Qwen-72B2025.05		20.42	20.46	6.53	7.78	88.25	87.76	82.26	81.03	76.15	76.56	68.12	67.64
GPT-4o2025.05		20.42	20.47	7.03	8.37	82.95	84.17	79.32	78.18	74.21	75.42	67.59	67.11
MBart-502025.05		19.93	24.91	8.56	9.94	76.58	88.95	84.96	84.15	73.31	77.75	66.77	68.28
SylAVL-CoT (Gemini 2.0)2025.05		19	18.49	6.79	8.67	76.52	78.59	72.89	69.53	71.42	72.78	66.35	65.67
Gemini 2.02025.05		18.57	17.85	6.73	8.26	78.13	79.04	75.44	73.27	71.69	72.34	66.86	66.04
Human Expert2025.05		18.04	16.22	7.55	8.01	63.92	61.26	57.46	53.74	68.56	68.67	64.86	63.83