Text Generation on WebText
37.5ROUGE-1NEFTune
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NEFTuneBase Model=GPT-Neo2026.05 | 37.5 | — | — | — | — | — | — | — | — | — | — | |
| Cont. (w/o DB)Base Model=GPT-Neo2026.05 | 37.5 | — | — | — | — | — | — | — | — | — | — | |
| Cont. (w/ DB)Base Model=GPT-Neo2026.05 | 37.5 | — | — | — | — | — | — | — | — | — | — | |
| VanillaBase Model=GPT-Neo2026.05 | 37.4 | — | — | — | — | — | — | — | — | — | — | |
| DiscreteBase Model=GPT-Neo2026.05 | 37.4 | — | — | — | — | — | — | — | — | — | — | |
| NEFTuneBase Model=Qwen32026.05 | 37.3 | — | — | — | — | — | — | — | — | — | — | |
| Cont. (w/ DB)Base Model=Qwen32026.05 | 37.3 | — | — | — | — | — | — | — | — | — | — | |
| Cont. (w/o DB)Base Model=Qwen32026.05 | 37.2 | — | — | — | — | — | — | — | — | — | — | |
| VanillaBase Model=Qwen32026.05 | 37.1 | — | — | — | — | — | — | — | — | — | — | |
| DiscreteBase Model=Qwen32026.05 | 37 | — | — | — | — | — | — | — | — | — | — | |
| Cont. (w/o DB)Base Model=OPT2026.05 | 36.9 | — | — | — | — | — | — | — | — | — | — | |
| Cont. (w/ DB)Base Model=OPT2026.05 | 36.9 | — | — | — | — | — | — | — | — | — | — | |
| NEFTuneBase Model=OPT2026.05 | 36.5 | — | — | — | — | — | — | — | — | — | — | |
| DiscreteBase Model=OPT2026.05 | 36.3 | — | — | — | — | — | — | — | — | — | — | |
| VanillaBase Model=OPT2026.05 | 36.2 | — | — | — | — | — | — | — | — | — | — | |
| Adversarial DecoderDecoding=Adversarial, Model=GPT-2 XL2021.02 | — | 0.05 | 0.03 | 0.07 | — | — | — | 0.06 | — | — | — | |
| Ancestral SamplingDecoding=Ancestral Sampling, Model=GPT-2 XL2021.02 | — | 19.3 | 0.02 | 0.02 | 0.69 | 0.37 | 11.4 | 0.88 | 9 | — | — | |
| Distinct-42021.02 | — | — | — | — | — | — | — | — | — | — | 0.524 | |
| Gen. PPL2021.02 | — | 0.643 | — | — | — | — | — | — | — | — | — | |
| GPT-2 LargeModel Size=Large, Decoding=Nucleus Sampling2021.02 | — | 0.9 | 0.02 | 0.03 | 0.68 | 0.37 | 14.9 | 0.936 | 12.6 | — | — | |
| GPT-2 MediumModel Size=Medium, Decoding=Nucleus Sampling2021.02 | — | 8.5 | 0 | 0.02 | 0.67 | 0.39 | 18.8 | 0.915 | -3.4 | — | — | |
| GPT-2 SmallModel Size=Small, Decoding=Nucleus Sampling2021.02 | — | 11.2 | 0.06 | 0.05 | 0.65 | 0.41 | 25.9 | 0.878 | -15.9 | — | — | |
| GPT-2 XLModel Size=XL, Decoding=Nucleus Sampling2021.02 | — | 1.5 | 0.01 | 0.03 | 0.69 | 0.36 | 13.7 | 0.94 | 15.7 | — | — | |
| Greedy SearchDecoding=Greedy, Model=GPT-2 XL2021.02 | — | 11.3 | 0.02 | 0.03 | 0.5 | 0.35 | 497 | 0.02 | — | — | — | |
| MAUVE2021.02 | — | — | — | — | — | — | — | 0.81 | — | — | — | |
| Nucleus SamplingDecoding=Nucleus Sampling, Model=GPT-2 XL2021.02 | — | 1.54 | 0.01 | 0.03 | 0.69 | 0.36 | 13.7 | 0.94 | 15.7 | — | — | |
| REP2021.02 | — | — | — | — | — | — | — | — | — | -0.143 | — | |
| Self-BLEU2021.02 | — | — | — | 0.405 | — | — | — | — | — | — | — | |
| Zipf Coef.2021.02 | — | — | 0.524 | — | — | — | — | — | — | — | — |