# Natural Language Generation on ToDs benchmark GPT-2 backbone (test)

Source: [Natural Language Generation on ToDs benchmark GPT-2 backbone (test)](<https://www.sota2.com/research/sota/natural-language-generation-on-tods-benchmark-gpt-2-backbone-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:41:26.361Z

Evaluation of Natural Language Generation performance on the test split of the ToDs benchmark, using a GPT-2 backbone configuration.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Natural Language Generation |
| Dataset (including any stated split) | ToDs benchmark GPT-2 backbone (test) |
| Metric | EER |
| Unit | PERCENT |
| Direction | Lower is better |
| Results loaded | 11 of 11 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [EER](<https://www.sota2.com/research/sota/natural-language-generation-on-tods-benchmark-gpt-2-backbone-test.md?metric=EER>): Lower is better; unit: PERCENT.

- [BLEU](<https://www.sota2.com/research/sota/natural-language-generation-on-tods-benchmark-gpt-2-backbone-test.md?metric=BLEU>): Higher is better; unit: PERCENT.

## Recorded results

| Method | EER | Result date | Report paper |
| --- | --- | --- | --- |
| MULTI | 3.42 | Not recorded | 68fa73471e209131cea5b810 |
| HESIT | 5.11 | Not recorded | 68fa73471e209131cea5b810 |
| GSS | 5.17 | Not recorded | 68fa73471e209131cea5b810 |
| UNIFORM | 5.42 | Not recorded | 68fa73471e209131cea5b810 |
| RANDOM | 5.82 | Not recorded | 68fa73471e209131cea5b810 |
| ARPER | 6.08 | Not recorded | 68fa73471e209131cea5b810 |
| VANILLA | 18.09 | Not recorded | 68fa73471e209131cea5b810 |
| EWC | 18.12 | Not recorded | 68fa73471e209131cea5b810 |
| L2 | 18.16 | Not recorded | 68fa73471e209131cea5b810 |
| LAMOL | 35.83 | Not recorded | 68fa73471e209131cea5b810 |
| A-GEM | 36.22 | Not recorded | 68fa73471e209131cea5b810 |

### MULTI

EER: 3.42. Result ID: 6992cf1b3a6c31484911a67d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

Recorded evaluation configuration: {"Memory size (Mem)":"Full data","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### HESIT

EER: 5.11. Result ID: 6992cf1b3a6c31484911a67c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

Recorded evaluation configuration: {"Memory size (Mem)":"t\|M\|","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### GSS

EER: 5.17. Result ID: 6992cf1b3a6c31484911a67a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66981e209131cea1132f.md>)

Recorded evaluation configuration: {"Memory size (Mem)":"t\|M\|","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### UNIFORM

EER: 5.42. Result ID: 6992cf1b3a6c31484911a679.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

Recorded evaluation configuration: {"Memory size (Mem)":"t\|M\|","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### RANDOM

EER: 5.82. Result ID: 6992cf1b3a6c31484911a678.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

Recorded evaluation configuration: {"Memory size (Mem)":"t\|M\|","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### ARPER

EER: 6.08. Result ID: 6992cf1b3a6c31484911a67b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa686f1e209131cea1c2b0.md>)

Recorded evaluation configuration: {"Memory size (Mem)":"t\|M\|","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### VANILLA

EER: 18.09. Result ID: 6992cf1b3a6c31484911a673.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

Recorded evaluation configuration: {"Memory size (Mem)":"0","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### EWC

EER: 18.12. Result ID: 6992cf1b3a6c31484911a675.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6cb31e209131cea3555e.md>)

Recorded evaluation configuration: {"Memory size (Mem)":"0","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### L2

EER: 18.16. Result ID: 6992cf1b3a6c31484911a674.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

Recorded evaluation configuration: {"Memory size (Mem)":"0","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### LAMOL

EER: 35.83. Result ID: 6992cf1b3a6c31484911a677.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66d51e209131cea12c0a.md>)

[Implementation](<https://github.com/jojotenya/LAMOL>)

Recorded evaluation configuration: {"Memory size (Mem)":"0","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

### A-GEM

EER: 36.22. Result ID: 6992cf1b3a6c31484911a676.

[Reporting paper](<https://www.sota2.com/research/paper/68fa73471e209131cea5b810.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65371e209131cea090ac.md>)

[Implementation](<https://github.com/facebookresearch/agem>)

Recorded evaluation configuration: {"Memory size (Mem)":"t\|M\|","Evaluation type":"Modularized"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.