Stylized Machine Translation on MSMT Average (test)
59Human Style AccuracySAE Cont.
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SAE Cont.Backbone=LLaMA 3.1 8B, alpha=52025.05 | 59 | — | 31 | 27 | 72 | 23.1 | |
| MSBackbone=LLaMA 3.1 8B, number of examples=202025.05 | 58 | — | 32 | 28 | 73 | 17.2 | |
| ZS-Exp.Backbone=LLaMA 3.1 8B2025.05 | 56 | — | 23 | 21 | 69 | 25.5 | |
| ActAddBackbone=LLaMA 3.1 8B2025.05 | 55 | — | 36 | 28 | 70 | 24.2 | |
| ReFTBackbone=LLaMA 3.1 8B2025.05 | 53 | — | 38 | 26 | 70 | 24.7 | |
| MSBackbone=Gemma 2 9B, number of examples=202025.05 | 48 | — | 31 | 27 | 73 | 16.8 | |
| ReFTBackbone=Gemma 2 9B2025.05 | 46 | — | 34 | 27 | 67 | 25.5 | |
| SAE Cont.Backbone=Gemma 2 9B, alpha=52025.05 | 46 | — | 33 | 29 | 72 | 25.1 | |
| ZS-Exp.Backbone=Gemma 2 9B2025.05 | 41 | — | 22 | 18 | 72 | 24.6 | |
| SAE Cont.Backbone=Gemma 2 2B, alpha=52025.05 | 39 | — | 27 | 19 | 70 | 41.1 | |
| MSBackbone=Gemma 2 2B, number of examples=202025.05 | 37 | — | 24 | 16 | 69 | 36 | |
| ActAddBackbone=Gemma 2 9B2025.05 | 32 | — | 24 | 20 | 70 | 25.3 | |
| ReFTBackbone=Gemma 2 2B2025.05 | 31 | — | 22 | 18 | 70 | 40.7 | |
| ZS-Exp.Backbone=Gemma 2 2B2025.05 | 30 | — | 22 | 16 | 68 | 41.3 | |
| ActAddBackbone=Gemma 2 2B2025.05 | 27 | — | 22 | 12 | 67 | 40.2 | |
| ZSBackbone=LLaMA 3.1 8B2025.05 | 24 | — | 8 | 5 | 70 | 25.9 | |
| ZSBackbone=Gemma 2 2B2025.05 | 21 | — | 10 | 5 | 69 | 39.8 | |
| ZSBackbone=Gemma 2 9B2025.05 | 15 | — | 8 | 4 | 71 | 25.7 | |
| Baselinetraining=raw parallel data2022.12 | — | 22.77 | — | — | — | — | |
| StyleAPmechanism=style prompt2022.12 | — | 26.2 | — | — | — | — | |
| Tag-tuningfine-tuning=explicit extra fine-tuning2022.12 | — | 25.11 | — | — | — | — |