Language Modeling on CODEPARROT
3.36PerplexityOURS (UNION)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OURS (UNION)Number of Parameters=1.3B2026.01 | 3.36 | — | |
| NO PROCEDURAL PRETRAININGNumber of Parameters=1.3B2026.01 | 3.45 | — | |
| OURS (UNION)Number of Parameters=350M2026.01 | 4.62 | — | |
| NO PROCEDURAL PRETRAININGNumber of Parameters=350M2026.01 | 4.97 | — | |
| Continuous Perturbation (w/o DB)Base Model=Qwen32026.05 | 22 | 45 | |
| Continuous Perturbation (w/ DB)Base Model=Qwen32026.05 | 22 | 44 | |
| NEFTuneBase Model=GPT-Neo2026.05 | 26 | 74 | |
| NEFTuneBase Model=Qwen32026.05 | 27 | 50 | |
| Continuous Perturbation (w/o DB)Base Model=GPT-Neo2026.05 | 28 | 57 | |
| Continuous Perturbation (w/ DB)Base Model=GPT-Neo2026.05 | 28 | 58 | |
| VanillaBase Model=Qwen32026.05 | 40 | 50 | |
| Discrete PerturbationBase Model=Qwen32026.05 | 43 | 36 | |
| VanillaBase Model=GPT-Neo2026.05 | 55 | 18 | |
| Discrete PerturbationBase Model=GPT-Neo2026.05 | 68 | 16 | |
| Continuous Perturbation (w/ DB)Base Model=OPT2026.05 | 289 | 26 | |
| Continuous Perturbation (w/o DB)Base Model=OPT2026.05 | 293 | 26 | |
| NEFTuneBase Model=OPT2026.05 | 464 | 23 | |
| VanillaBase Model=OPT2026.05 | 602 | 17 | |
| Discrete PerturbationBase Model=OPT2026.05 | 751 | 14 |