Stage-aware Prefill
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Stage-aware Prefill
63.29TTFT (ms)
32
Stage-aware Prefill 128K Prompt
38.64TPOT (ms/token)
4
Stage-aware Prefill 64K Prompt
25.52TPOT (ms/token)
4
Stage-aware Prefill 32K Prompt
24.03Latency (ms/token)
4
Stage-aware Prefill 16K Prompt
23.97TPOT (ms/token)
4
Stage-aware Prefill 8K Prompt
24.14TPOT (ms/token)
4
Stage-aware Prefill 4K Prompt
24.15TPOT (ms/token)
4
Stage-aware Prefill 2K Prompt
24.11TPOT (ms/token)
4
Stage-aware Prefill 1K Prompt
24.05TPOT (ms/token)
4