Dialect Translation on Jeolla, Gyeongsang, and Jeju dialects averaged (test)
71.31chrF++ICL
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| ICLBackbone=Llama-3.12025.11 | 71.31 | 61.75 | -0.26 | 15 | |
| ICLBackbone=EEVE2025.11 | 70.99 | 61.34 | -0.26 | 16 | |
| ICL + DIA-REFINE(S)Backbone=EEVE2025.11 | 70.48 | 60.68 | -0.23 | 19 | |
| ICL + DIA-REFINE(S)Backbone=Llama-3.12025.11 | 70.42 | 60.57 | -0.24 | 18 | |
| ZSBackbone=Llama-3.12025.11 | 68.97 | 58.53 | -0.37 | 3 | |
| ICL + DIA-REFINE(M)Backbone=Llama-3.12025.11 | 66.15 | 54.67 | -0.1 | 36 | |
| ICLBackbone=Model Avg2025.11 | 65.99 | 54.25 | -0.11 | 33 | |
| ICL + DIA-REFINE(M)Backbone=EEVE2025.11 | 65.67 | 54.3 | -0.05 | 42 | |
| ICLBackbone=EXAONE-3.52025.11 | 65.41 | 52.88 | -0.04 | 41 | |
| ICL + DIA-REFINE(S)Backbone=Model Avg2025.11 | 63.72 | 51.23 | -0.06 | 42 | |
| ICLBackbone=Gemini-1.52025.11 | 63.08 | 49.1 | 0.31 | 85 | |
| ZS + DIA-REFINE(S)Backbone=Llama-3.12025.11 | 62.52 | 49.82 | -0.37 | 3 | |
| ICL + DIA-REFINE(S)Backbone=Gemini-1.52025.11 | 62.13 | 47.73 | 0.36 | 97 | |
| ICL + DIA-REFINE(M)Backbone=Gemini-1.52025.11 | 61.68 | 47.22 | 0.39 | 99 | |
| ZS + DIA-REFINE(S)Backbone=HyperCLOVAX, Prompting=Zero-shot, Refinement=DIA-REFINE (Single)2025.11 | 60.76 | 47.94 | -0.37 | 4 | |
| ZSBackbone=HyperCLOVAX, Prompting=Zero-shot2025.11 | 60.33 | 47.64 | -0.37 | 3 | |
| ICL + DIA-REFINE(S)Backbone=EXAONE-3.52025.11 | 60.19 | 45.75 | 0.07 | 60 | |
| ICLBackbone=HyperCLOVAX, Prompting=In-context Learning2025.11 | 59.17 | 46.21 | -0.3 | 9 | |
| ICL + DIA-REFINE(M)Backbone=Model Avg2025.11 | 58.89 | 45.41 | 0.09 | 62 | |
| ICL + DIA-REFINE(S)Backbone=HyperCLOVAX, Prompting=In-context Learning, Refinement=DIA-REFINE (Single)2025.11 | 55.35 | 41.44 | -0.24 | 15 | |
| ICL + DIA-REFINE(M)Backbone=EXAONE-3.52025.11 | 54.95 | 39.14 | 0.27 | 94 | |
| ZSBackbone=EEVE2025.11 | 54.03 | 41.71 | -0.38 | 2 | |
| ZS + DIA-REFINE(M)Backbone=HyperCLOVAX, Prompting=Zero-shot, Refinement=DIA-REFINE (Multi)2025.11 | 52.52 | 38.82 | -0.32 | 8 | |
| ZS + DIA-REFINE(S)Backbone=EEVE2025.11 | 52.41 | 39.49 | -0.37 | 3 | |
| ZSBackbone=Model Avg2025.11 | 50.38 | 36.48 | -0.23 | 20 | |
| ZS + DIA-REFINE(S)Backbone=Model Avg2025.11 | 48.73 | 34.23 | -0.21 | 23 | |
| ICL + DIA-REFINE(M)Backbone=HyperCLOVAX, Prompting=In-context Learning, Refinement=DIA-REFINE (Multi)2025.11 | 46 | 31.7 | -0.06 | 41 | |
| ZS + DIA-REFINE(M)Backbone=Llama-3.12025.11 | 44.29 | 28.98 | -0.32 | 8 | |
| ZS + DIA-REFINE(M)Backbone=Model Avg2025.11 | 40.08 | 24.66 | -0.12 | 34 | |
| ZS + DIA-REFINE(M)Backbone=EEVE2025.11 | 39.82 | 25.24 | -0.28 | 13 | |
| ZSBackbone=Gemini-1.52025.11 | 39.5 | 23.06 | 0.36 | 93 | |
| ZS + DIA-REFINE(M)Backbone=Gemini-1.52025.11 | 38.55 | 22.13 | 0.4 | 99 | |
| ZS + DIA-REFINE(S)Backbone=Gemini-1.52025.11 | 38.03 | 21.92 | 0.38 | 98 | |
| ZS + DIA-REFINE(S)Backbone=EXAONE-3.52025.11 | 29.94 | 11.97 | -0.32 | 8 | |
| ZSBackbone=EXAONE-3.52025.11 | 29.05 | 11.44 | -0.39 | 1 | |
| ZS + DIA-REFINE(M)Backbone=EXAONE-3.52025.11 | 25.2 | 8.13 | -0.08 | 42 |