Language Modeling on Standard NLP Benchmarks
5.61PPLChunkWise LoRA
Evaluation results
| Method | Links | |
|---|---|---|
| ChunkWise LoRALatency=14.9, Memory=9.12026.01 | 5.61 | |
| AdaLoRALatency=17.8, Memory=10.52026.01 | 5.66 | |
| LoRArank=32, Latency=21.7, Memory=13.92026.01 | 5.69 | |
| LoRArank=16, Latency=20.1, Memory=12.72026.01 | 5.74 | |
| Vanilla LLaMA-7BLatency=22.5, Memory=14.62026.01 | 5.84 | |
| LoRArank=8, Latency=19.3, Memory=11.22026.01 | 5.97 |