Language Modeling on WebText
58MauveEMO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EMOModel=GPT-22026.05 | 58 | 38 | — | |
| MLE (ours)Model=GPT-Neo-1.3B2026.05 | 58 | 39 | — | |
| MLE (ours)Model=GPT-22026.05 | 57 | 39 | — | |
| MLE (ours)Model=OPT-125M2026.05 | 56 | 39 | — | |
| EMOModel=GPT-Neo-1.3B2026.05 | 56 | 36 | — | |
| MLEModel=GPT-Neo-1.3B2026.05 | 54 | 38 | — | |
| MixCEModel=GPT-Neo-1.3B2026.05 | 53 | 38 | — | |
| TVDModel=GPT-Neo-1.3B2026.05 | 53 | 38 | — | |
| BrierModel=GPT-Neo-1.3B2026.05 | 51 | 37 | — | |
| EMOModel=OPT-125M2026.05 | 49 | 37 | — | |
| BrierModel=GPT-22026.05 | 49 | 36 | — | |
| MixCEModel=GPT-22026.05 | 48 | 37 | — | |
| MLEModel=GPT-22026.05 | 47 | 37 | — | |
| MixCEModel=OPT-125M2026.05 | 46 | 37 | — | |
| TVDModel=GPT-22026.05 | 45 | 37 | — | |
| MLEModel=OPT-125M2026.05 | 44 | 37 | — | |
| TVDModel=OPT-125M2026.05 | 40 | 37 | — | |
| BrierModel=OPT-125M2026.05 | 40 | 36 | — | |
| Continuous Perturbation (w/o DB)Base Model=Qwen32026.05 | 0.68 | — | 99 | |
| Continuous Perturbation (w/o DB)Base Model=GPT-Neo2026.05 | 0.68 | — | 66 | |
| Continuous Perturbation (w/ DB)Base Model=GPT-Neo2026.05 | 0.68 | — | 66 | |
| Continuous Perturbation (w/ DB)Base Model=Qwen32026.05 | 0.67 | — | 99 | |
| NEFTuneBase Model=GPT-Neo2026.05 | 0.67 | — | 68 | |
| Continuous Perturbation (w/ DB)Base Model=OPT2026.05 | 0.66 | — | 104 | |
| Continuous Perturbation (w/o DB)Base Model=OPT2026.05 | 0.65 | — | 106 | |
| NEFTuneBase Model=Qwen32026.05 | 0.58 | — | 129 | |
| VanillaBase Model=GPT-Neo2026.05 | 0.47 | — | 110 | |
| Discrete PerturbationBase Model=GPT-Neo2026.05 | 0.45 | — | 131 | |
| VanillaBase Model=Qwen32026.05 | 0.43 | — | 156 | |
| Discrete PerturbationBase Model=Qwen32026.05 | 0.38 | — | 191 | |
| NEFTuneBase Model=OPT2026.05 | 0.37 | — | 204 | |
| Discrete PerturbationBase Model=OPT2026.05 | 0.34 | — | 251 | |
| VanillaBase Model=OPT2026.05 | 0.33 | — | 226 |