End-to-end Time-to-First-Token (TTFT) on Long-context sequences
257TTFT (ms)FlashPrefill
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FlashPrefillModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=4K2026.03 | 257 | 1.04 | |
| FullModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=4K2026.03 | 267 | — | |
| FlashPrefillModel=Qwen2.5-7B-Instruct, Sequence Length=4K2026.03 | 406 | 1.04 | |
| FullModel=Qwen2.5-7B-Instruct, Sequence Length=4K2026.03 | 423 | — | |
| FlashPrefillModel=Llama-3.1-8B-Instruct, Sequence Length=4K2026.03 | 447 | 1.05 | |
| FullModel=Llama-3.1-8B-Instruct, Sequence Length=4K2026.03 | 471 | — | |
| FlashPrefillModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=8K2026.03 | 519 | 1.18 | |
| FullModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=8K2026.03 | 613 | — | |
| FlashPrefillModel=Qwen2.5-7B-Instruct, Sequence Length=8K2026.03 | 851 | 1.07 | |
| FullModel=Qwen2.5-7B-Instruct, Sequence Length=8K2026.03 | 912 | — | |
| FlashPrefillModel=Llama-3.1-8B-Instruct, Sequence Length=8K2026.03 | 913 | 1.07 | |
| FullModel=Llama-3.1-8B-Instruct, Sequence Length=8K2026.03 | 976 | — | |
| FlashPrefillModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=16K2026.03 | 1,090 | 1.47 | |
| FullModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=16K2026.03 | 1,603 | — | |
| FlashPrefillModel=Qwen2.5-7B-Instruct, Sequence Length=16K2026.03 | 1,707 | 1.2 | |
| FlashPrefillModel=Llama-3.1-8B-Instruct, Sequence Length=16K2026.03 | 1,860 | 1.18 | |
| FullModel=Qwen2.5-7B-Instruct, Sequence Length=16K2026.03 | 2,048 | — | |
| FullModel=Llama-3.1-8B-Instruct, Sequence Length=16K2026.03 | 2,189 | — | |
| FlashPrefillModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=32K2026.03 | 2,331 | 1.98 | |
| FlashPrefillModel=Qwen2.5-7B-Instruct, Sequence Length=32K2026.03 | 3,534 | 1.34 | |
| FlashPrefillModel=Llama-3.1-8B-Instruct, Sequence Length=32K2026.03 | 3,618 | 1.49 | |
| FullModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=32K2026.03 | 4,635 | — | |
| FullModel=Qwen2.5-7B-Instruct, Sequence Length=32K2026.03 | 4,735 | — | |
| FlashPrefillModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=64K2026.03 | 4,909 | 3.1 | |
| FullModel=Llama-3.1-8B-Instruct, Sequence Length=32K2026.03 | 5,398 | — | |
| FlashPrefillModel=Qwen2.5-7B-Instruct, Sequence Length=64K2026.03 | 7,323 | 1.72 | |
| FlashPrefillModel=Llama-3.1-8B-Instruct, Sequence Length=64K2026.03 | 7,481 | 1.97 | |
| FlashPrefillModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=128K2026.03 | 10,702 | 5.02 | |
| FullModel=Qwen2.5-7B-Instruct, Sequence Length=64K2026.03 | 12,563 | — | |
| FullModel=Llama-3.1-8B-Instruct, Sequence Length=64K2026.03 | 14,774 | — | |
| FlashPrefillModel=Llama-3.1-8B-Instruct, Sequence Length=128K2026.03 | 15,061 | 3.02 | |
| FullModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=64K2026.03 | 15,203 | — | |
| FlashPrefillModel=Qwen2.5-7B-Instruct, Sequence Length=128K2026.03 | 15,213 | 2.45 | |
| FullModel=Qwen2.5-7B-Instruct, Sequence Length=128K2026.03 | 37,239 | — | |
| FullModel=Llama-3.1-8B-Instruct, Sequence Length=128K2026.03 | 45,464 | — | |
| FullModel=Qwen3-30B-A3B-Instruct-2507, Sequence Length=128K2026.03 | 53,752 | — |