Language Modeling on C4-en LLaMA-1B (13.1B tokens)
14.05Perplexity (PPL)CR-Net
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| CR-NetAlignment Target=Memory overhead2025.09 | 14.05 | 870 | 6.48 | |
| Apollo2025.09 | 14.2 | 1,339 | 6.6 | |
| LORO2025.09 | 15.19 | 609 | 4.54 | |
| CR-NetAlignment Target=Parameter complexity2025.09 | 15.22 | 583 | 4.35 | |
| CoLA2025.09 | 15.52 | 609 | 4.54 | |
| Full-rank2025.09 | 15.56 | 1,339 | 9.98 | |
| GaLore2025.09 | 15.64 | 1,339 | 6.6 | |
| RSO2025.09 | 15.86 | 1,339 | 6.6 | |
| SLTrain2025.09 | 16.14 | 646 | 4.81 | |
| ReLoRA2025.09 | 18.33 | 1,339 | 6.79 | |
| LoRA2025.09 | 19.21 | 1,339 | 6.79 |