LLM Decoding on ShareGPT
2.4Latency (ms/token)Llama2-7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama2-7BBackbone=Llama2-7B2026.03 | 2.4 | 12.9 | |
| AdaFuseBackbone=Llama2-7B, Adapter Granularity=token-wise2026.03 | 3.1 | 13.8 | |
| PESCBackbone=Llama2-7B, Adapter Granularity=block-wise2026.03 | 8.5 | 13.1 | |
| MoRALBackbone=Llama2-7B, Adapter Granularity=layer-wise2026.03 | 8.6 | 13.3 | |
| MOLABackbone=Llama2-7B, Adapter Granularity=layer-wise2026.03 | 25.3 | 26.3 |