# Code-switched Text Generation on EN-CS Out of domain (test)

Source: [Code-switched Text Generation on EN-CS Out of domain (test)](<https://www.sota2.com/research/sota/code-switched-text-generation-on-en-cs-out-of-domain-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-09T06:39:14.357Z

Evaluation of Code-switched Text Generation performance on the out-of-domain test split of the EN-CS dataset, measured using BLEU, BERTScore, and chrF.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Code-switched Text Generation |
| Dataset (including any stated split) | EN-CS Out of domain (test) |
| Metric | BLEU |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [BLEU](<https://www.sota2.com/research/sota/code-switched-text-generation-on-en-cs-out-of-domain-test.md?metric=BLEU>): Higher is better; unit: PERCENT.

- [BERTScore](<https://www.sota2.com/research/sota/code-switched-text-generation-on-en-cs-out-of-domain-test.md?metric=BERTScore>): Higher is better; unit: PERCENT.

- [chrF](<https://www.sota2.com/research/sota/code-switched-text-generation-on-en-cs-out-of-domain-test.md?metric=chrF>): Higher is better; unit: PERCENT.

## Recorded results

| Method | BLEU | Result date | Report paper |
| --- | --- | --- | --- |
| Identity | 41.54 | Not recorded | 68fa78c51e209131cea7a6a5 |
| GPT-4o | 15.65 | Not recorded | 68fa78c51e209131cea7a6a5 |
| Llama3 8B | 14.93 | Not recorded | 68fa78c51e209131cea7a6a5 |
| Llama3 8B Instruct | 14.59 | Not recorded | 68fa78c51e209131cea7a6a5 |
| NLLB | 14.45 | Not recorded | 68fa78c51e209131cea7a6a5 |
| Llama3.3-70B | 14.1 | Not recorded | 68fa78c51e209131cea7a6a5 |

### Identity

BLEU: 41.54. Result ID: 69ae6aaa6dbb7075b955d999.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

Recorded evaluation configuration: {"Model":"Identity"}. This record does not assert independent reproduction.

### GPT-4o

BLEU: 15.65. Result ID: 69ae6aaa6dbb7075b955d997.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

Recorded evaluation configuration: {"Model":"GPT-4ofs","mode":"few-shot"}. This record does not assert independent reproduction.

### Llama3 8B

BLEU: 14.93. Result ID: 69ae6aaa6dbb7075b955d994.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

Recorded evaluation configuration: {"Model":"Llama3 8B","Status":"Fine-tuned"}. This record does not assert independent reproduction.

### Llama3 8B Instruct

BLEU: 14.59. Result ID: 69ae6aaa6dbb7075b955d995.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

Recorded evaluation configuration: {"Model":"Llama3 8B Instruct","Status":"Fine-tuned"}. This record does not assert independent reproduction.

### NLLB

BLEU: 14.45. Result ID: 69ae6aaa6dbb7075b955d998.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

Recorded evaluation configuration: {"Model":"NLLB"}. This record does not assert independent reproduction.

### Llama3.3-70B

BLEU: 14.1. Result ID: 69ae6aaa6dbb7075b955d996.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78c51e209131cea7a6a5.md>)

Recorded evaluation configuration: {"Model":"Llama3.3-70Bfs","mode":"few-shot"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.