Language Modeling on GPT-2 1,000 samples
27.99PPLDirect replacement
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Direct replacementParams (%)=28.5%2026.01 | 27.99 | 40.7 | 4.44 | 167.6 | |
| Full fine-tuningParams (%)=100%2026.01 | 35.5 | 31.2 | 5.51 | 189.7 | |
| LoRArank=8, Params (%)=0.24%2026.01 | 38.51 | 32.1 | 4.96 | 175.5 | |
| Bridge-mediatedParams (%)=18.0%2026.01 | 38.68 | 27.7 | 4.5 | 182.7 |