Time-to-first-token (TTFT) on Llama-3.1-8B-Instruct 96K input length
10.9TTFT (s)TriangleMix + MInference
Evaluation Results
| Method | Links | |
|---|---|---|
| TriangleMix + MInference2025.07 | 10.9 | |
| TriangleMix + FlexPrefill2025.07 | 11.3 | |
| FlexPrefill2025.07 | 12.3 | |
| TriangleMix + XAttention2025.07 | 12.4 | |
| MInference2025.07 | 13.4 | |
| TriangleMix2025.07 | 15.2 | |
| DuoAttention2025.07 | 15.5 | |
| XAttention2025.07 | 15.7 | |
| Dense2025.07 | 21.3 |