Language Modeling on WikiText-103 (PPL and Mauve)
72Perplexity (PPL)Continuous Perturbation (w/o DB)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Continuous Perturbation (w/o DB)Base Model=Qwen32026.05 | 72 | 51 | |
| Continuous Perturbation (w/ DB)Base Model=Qwen32026.05 | 73 | 50 | |
| Continuous Perturbation (w/ DB)Base Model=GPT-Neo2026.05 | 76 | 68 | |
| NEFTuneBase Model=GPT-Neo2026.05 | 77 | 69 | |
| Continuous Perturbation (w/o DB)Base Model=GPT-Neo2026.05 | 77 | 68 | |
| VanillaBase Model=GPT-Neo2026.05 | 78 | 83 | |
| Discrete PerturbationBase Model=GPT-Neo2026.05 | 87 | 81 | |
| NEFTuneBase Model=Qwen32026.05 | 95 | 77 | |
| VanillaBase Model=Qwen32026.05 | 97 | 76 | |
| Discrete PerturbationBase Model=Qwen32026.05 | 113 | 75 | |
| Continuous Perturbation (w/ DB)Base Model=OPT2026.05 | 129 | 62 | |
| Continuous Perturbation (w/o DB)Base Model=OPT2026.05 | 132 | 60 | |
| VanillaBase Model=OPT2026.05 | 145 | 77 | |
| NEFTuneBase Model=OPT2026.05 | 150 | 79 | |
| Discrete PerturbationBase Model=OPT2026.05 | 159 | 79 |