Language Modeling Generation Metrics on WikiText-2
0.9MauveMLE (ours)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MLE (ours)Model=OPT-125M2026.05 | 0.9 | 41 | — | |
| MLE (ours)Model=GPT-22026.05 | 0.89 | 40 | — | |
| MLE (ours)Model=GPT-Neo-1.3B2026.05 | 0.87 | 41 | — | |
| EMOModel=GPT-Neo-1.3B2026.05 | 0.85 | 38 | — | |
| MLEModel=OPT-125M2026.05 | 0.83 | 38 | — | |
| VanillaBase Model=GPT-Neo2026.05 | 0.83 | — | 78 | |
| EMOModel=GPT-22026.05 | 0.82 | 39 | — | |
| MLEModel=GPT-Neo-1.3B2026.05 | 0.82 | 39 | — | |
| TVDModel=GPT-Neo-1.3B2026.05 | 0.81 | 39 | — | |
| Discrete PerturbationBase Model=GPT-Neo2026.05 | 0.81 | — | 87 | |
| EMOModel=OPT-125M2026.05 | 0.8 | 38 | — | |
| MixCEModel=GPT-Neo-1.3B2026.05 | 0.79 | 39 | — | |
| BrierModel=GPT-Neo-1.3B2026.05 | 0.79 | 38 | — | |
| Discrete PerturbationBase Model=OPT2026.05 | 0.79 | — | 159 | |
| NEFTuneBase Model=OPT2026.05 | 0.79 | — | 150 | |
| TVDModel=OPT-125M2026.05 | 0.78 | 38 | — | |
| MixCEModel=OPT-125M2026.05 | 0.77 | 38 | — | |
| MixCEModel=GPT-22026.05 | 0.77 | 38 | — | |
| VanillaBase Model=OPT2026.05 | 0.77 | — | 145 | |
| NEFTuneBase Model=Qwen32026.05 | 0.77 | — | 95 | |
| VanillaBase Model=Qwen32026.05 | 0.76 | — | 97 | |
| TVDModel=GPT-22026.05 | 0.75 | 37 | — | |
| Discrete PerturbationBase Model=Qwen32026.05 | 0.75 | — | 113 | |
| MLEModel=GPT-22026.05 | 0.74 | 37 | — | |
| BrierModel=OPT-125M2026.05 | 0.71 | 37 | — | |
| BrierModel=GPT-22026.05 | 0.71 | 36 | — | |
| NEFTuneBase Model=GPT-Neo2026.05 | 0.69 | — | 77 | |
| Continuous Perturbation (w/o DB)Base Model=GPT-Neo2026.05 | 0.68 | — | 77 | |
| Continuous Perturbation (w/ DB)Base Model=GPT-Neo2026.05 | 0.68 | — | 76 | |
| Continuous Perturbation (w/ DB)Base Model=OPT2026.05 | 0.62 | — | 129 | |
| Continuous Perturbation (w/o DB)Base Model=OPT2026.05 | 0.6 | — | 132 | |
| Continuous Perturbation (w/o DB)Base Model=Qwen32026.05 | 0.51 | — | 72 | |
| Continuous Perturbation (w/ DB)Base Model=Qwen32026.05 | 0.5 | — | 73 |