Text Style Transfer on IRS style domain
49.5BLEUSimilar 5-shot finetuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Similar 5-shot finetuningBackbone=Llama-3.1-8B-Instruct, Number of shots=5, Retrieval strategy=Similar (cosine distance), RAG method=Finetuning, Terminology and name retrieval=false2026.02 | 49.5 | 79.6 | |
| Similar 5-shot finetuning w/ terminology and name retrievalBackbone=Llama-3.1-8B-Instruct, Number of shots=5, Retrieval strategy=Similar (cosine distance), RAG method=Finetuning, Terminology and name retrieval=true2026.02 | 49.28 | 89.5 | |
| Random 5-shot finetuningBackbone=Llama-3.1-8B-Instruct, Number of shots=5, Retrieval strategy=Random, RAG method=Finetuning, Terminology and name retrieval=false2026.02 | 48.89 | 82.6 | |
| Similar 3-shot finetuningBackbone=Llama-3.1-8B-Instruct, Number of shots=3, Retrieval strategy=Similar (cosine distance), RAG method=Finetuning, Terminology and name retrieval=false2026.02 | 47.79 | 74.9 | |
| Random 3-shot finetuningBackbone=Llama-3.1-8B-Instruct, Number of shots=3, Retrieval strategy=Random, RAG method=Finetuning, Terminology and name retrieval=false2026.02 | 47.23 | 83.9 | |
| Zero-shot finetuningBackbone=Llama-3.1-8B-Instruct, Number of shots=0, RAG method=Finetuning, Terminology and name retrieval=false2026.02 | 42.39 | 79.3 | |
| APE with MarianBackbone=Marian, RAG method=Automatic Post-Editing2026.02 | 36.81 | 64.2 | |
| 5-shot ICL w/ terminology and name retrievalBackbone=Llama-3.1-8B-Instruct, Number of shots=5, RAG method=In-Context Learning, Terminology and name retrieval=true2026.02 | 28.53 | 67.2 | |
| 5-shot ICLBackbone=Llama-3.1-8B-Instruct, Number of shots=5, RAG method=In-Context Learning, Terminology and name retrieval=false2026.02 | 27.79 | 59.1 |