# Topic Classification and Text Generation on AGNews (test)

Source: [Topic Classification and Text Generation on AGNews (test)](<https://www.sota2.com/research/sota/topic-classification-and-text-generation-on-agnews-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:55:11.923Z

Evaluation of AGNews (test) covering both Topic Classification performance (Accuracy, F1) and associated Text Generation quality using language model metrics (PPL, S-BLEU).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Topic Classification and Text Generation |
| Dataset (including any stated split) | AGNews (test) |
| Metric | PPL (Output) |
| Unit | SCALAR |
| Direction | Lower is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [PPL (Output)](<https://www.sota2.com/research/sota/topic-classification-and-text-generation-on-agnews-test.md?metric=PPL%20(Output)>): Lower is better; unit: SCALAR.

- [PPL (Model)](<https://www.sota2.com/research/sota/topic-classification-and-text-generation-on-agnews-test.md?metric=PPL%20(Model)>): Lower is better; unit: SCALAR.

- [Accuracy](<https://www.sota2.com/research/sota/topic-classification-and-text-generation-on-agnews-test.md?metric=Accuracy>): Higher is better; unit: PERCENT.

- [F1 Score](<https://www.sota2.com/research/sota/topic-classification-and-text-generation-on-agnews-test.md?metric=F1%20Score>): Higher is better; unit: PERCENT.

- [AUC](<https://www.sota2.com/research/sota/topic-classification-and-text-generation-on-agnews-test.md?metric=AUC>): Higher is better; unit: PERCENT.

- [Diversity](<https://www.sota2.com/research/sota/topic-classification-and-text-generation-on-agnews-test.md?metric=Diversity>): Higher is better; unit: PERCENT.

- [S-BLEU](<https://www.sota2.com/research/sota/topic-classification-and-text-generation-on-agnews-test.md?metric=S-BLEU>): Higher is better; unit: PERCENT.

## Recorded results

| Method | PPL (Output) | Result date | Report paper |
| --- | --- | --- | --- |
| GPT2(raw) | 16.94 | Not recorded | 68fa6fa11e209131cea46ca8 |
| PF | 20.27 | Not recorded | 68fa6fa11e209131cea46ca8 |
| Ctr-PF | 20.41 | Not recorded | 68fa6fa11e209131cea46ca8 |
| GPT2 | 22.22 | Not recorded | 68fa6fa11e209131cea46ca8 |
| +PT | 23.74 | Not recorded | 68fa6fa11e209131cea46ca8 |
| +PT+noise | 26.39 | Not recorded | 68fa6fa11e209131cea46ca8 |
| +PT(noise)+PL | 30.62 | Not recorded | 68fa6fa11e209131cea46ca8 |
| Test set | 31.04 | Not recorded | 68fa6fa11e209131cea46ca8 |
| +PT(select)+PL | 31.34 | Not recorded | 68fa6fa11e209131cea46ca8 |
| +PT(noise)+PL | 32.36 | Not recorded | 68fa6fa11e209131cea46ca8 |
| +PT(select)+PL | 33.23 | Not recorded | 68fa6fa11e209131cea46ca8 |
| DuNST | 34.73 | Not recorded | 68fa6fa11e209131cea46ca8 |
| T5 | 48.33 | Not recorded | 68fa6fa11e209131cea46ca8 |
| UniLM | 55.79 | Not recorded | 68fa6fa11e209131cea46ca8 |
| +PT | 57.4 | Not recorded | 68fa6fa11e209131cea46ca8 |
| +PT+noise | 58.59 | Not recorded | 68fa6fa11e209131cea46ca8 |

### GPT2(raw)

PPL (Output): 16.94. Result ID: 6990e52d3a6c314849041863.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

Recorded evaluation configuration: {"Mode":"Raw"}. This record does not assert independent reproduction.

### PF

PPL (Output): 20.27. Result ID: 6990e52d3a6c314849041867.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa690f1e209131cea1fe0e.md>)

Recorded evaluation configuration: {"Protocol":"Lightweight method"}. This record does not assert independent reproduction.

### Ctr-PF

PPL (Output): 20.41. Result ID: 6990e52d3a6c314849041868.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6bdd1e209131cea30afd.md>)

Recorded evaluation configuration: {"Protocol":"Lightweight method"}. This record does not assert independent reproduction.

### GPT2

PPL (Output): 22.22. Result ID: 6990e52d3a6c314849041864.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

Recorded evaluation configuration: {"Protocol":"Finetune LM"}. This record does not assert independent reproduction.

### +PT

PPL (Output): 23.74. Result ID: 6990e52d3a6c314849041869.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

Recorded evaluation configuration: {"Protocol":"Self-Training","Base Model":"GPT2"}. This record does not assert independent reproduction.

### +PT+noise

PPL (Output): 26.39. Result ID: 6990e52d3a6c31484904186a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa688b1e209131cea1cd36.md>)

Recorded evaluation configuration: {"Protocol":"Self-Training","Base Model":"GPT2"}. This record does not assert independent reproduction.

### +PT(noise)+PL

PPL (Output): 30.62. Result ID: 6990e52d3a6c31484904186b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa688b1e209131cea1cd36.md>)

Recorded evaluation configuration: {"Protocol":"Self-Training","Base Model":"GPT2"}. This record does not assert independent reproduction.

### Test set

PPL (Output): 31.04. Result ID: 6990e52d3a6c314849041862.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### +PT(select)+PL

PPL (Output): 31.34. Result ID: 6990e52d3a6c31484904186c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

Recorded evaluation configuration: {"Protocol":"Self-Training","Base Model":"GPT2"}. This record does not assert independent reproduction.

### +PT(noise)+PL

PPL (Output): 32.36. Result ID: 6990e52d3a6c31484904186f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa688b1e209131cea1cd36.md>)

Recorded evaluation configuration: {"Protocol":"Self-Training","Base Model":"UniLM"}. This record does not assert independent reproduction.

### +PT(select)+PL

PPL (Output): 33.23. Result ID: 6990e52d3a6c314849041870.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

Recorded evaluation configuration: {"Protocol":"Self-Training","Base Model":"UniLM"}. This record does not assert independent reproduction.

### DuNST

PPL (Output): 34.73. Result ID: 6990e52d3a6c314849041871.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Implementation](<https://github.com/peterfengyx/DuNST>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### T5

PPL (Output): 48.33. Result ID: 6990e52d3a6c314849041866.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa709e1e209131cea4c470.md>)

[Implementation](<https://github.com/google-research/text-to-text-transfer-transformer>)

Recorded evaluation configuration: {"Protocol":"Finetune LM"}. This record does not assert independent reproduction.

### UniLM

PPL (Output): 55.79. Result ID: 6990e52d3a6c314849041865.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66831e209131cea10caa.md>)

[Implementation](<https://github.com/microsoft/unilm>)

Recorded evaluation configuration: {"Protocol":"Finetune LM"}. This record does not assert independent reproduction.

### +PT

PPL (Output): 57.4. Result ID: 6990e52d3a6c31484904186d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

Recorded evaluation configuration: {"Protocol":"Self-Training","Base Model":"UniLM"}. This record does not assert independent reproduction.

### +PT+noise

PPL (Output): 58.59. Result ID: 6990e52d3a6c31484904186e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6fa11e209131cea46ca8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa688b1e209131cea1cd36.md>)

Recorded evaluation configuration: {"Protocol":"Self-Training","Base Model":"UniLM"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
