Time-to-first-token (TTFT) on Llama 8B Instruct 32K input length 3.1
3.4TTFT (s)TriangleMix + FlexPrefill
Evaluation Results
| Method | Links | |
|---|---|---|
| TriangleMix + FlexPrefill2025.07 | 3.4 | |
| TriangleMix + XAttention2025.07 | 3.5 | |
| DuoAttention2025.07 | 3.6 | |
| TriangleMix2025.07 | 3.6 | |
| FlexPrefill2025.07 | 3.6 | |
| Dense2025.07 | 4.1 | |
| XAttention2025.07 | 4.1 | |
| TriangleMix + MInference2025.07 | 4.2 | |
| MInference2025.07 | 5.5 |