LLM Generation Efficiency on Qwen 2.5 14B (2048 input + 8192 generation tokens)
287.7End-to-end Latency (s)GRIFFIN
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRIFFINSparsity=50%, r=256, Hardware=NVIDIA L40 GPU, Precision=BF16, Model Portion=First half of the model2025.05 | 287.7 | 35 | |
| CapreseSparsity=50%, r=256, Hardware=NVIDIA L40 GPU, Precision=BF162025.05 | 288.8 | 35 | |
| LoRASparsity=50%, r=256, Hardware=NVIDIA L40 GPU, Precision=BF16, Configuration=GRIFFIN paired with LoRA weights2025.05 | 312.5 | 38 | |
| FullHardware=NVIDIA L40 GPU, Precision=BF162025.05 | 344.9 | 42 |