Time to First Token (TTFT) on NVFP4-W4A16
182Time to First Token (ms)FlashRT
Evaluation Results
| Method | Links | |
|---|---|---|
| FlashRTHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=4096, Reuse Strategy=with reuse (capsule)2026.06 | 182 | |
| FlashRTHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=2048, Reuse Strategy=with reuse (capsule)2026.06 | 183 | |
| FlashRTHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=8192, Reuse Strategy=with reuse (capsule)2026.06 | 192 | |
| FlashRTHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=16384, Reuse Strategy=with reuse (capsule)2026.06 | 202 | |
| FlashRTHardware=Jetson AGX Thor (SM110), Prefix length=4096, Reuse condition=with reuse (capsule)2026.06 | 225 | |
| FlashRTHardware=Jetson AGX Thor (SM110), Prefix length=16384, Reuse condition=with reuse (capsule)2026.06 | 236 | |
| FlashRTHardware=Jetson AGX Thor (SM110), Prefix length=2048, Reuse condition=with reuse (capsule)2026.06 | 240 | |
| FlashRTHardware=Jetson AGX Thor (SM110), Prefix length=8192, Reuse condition=with reuse (capsule)2026.06 | 244 | |
| vLLMHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=4096, Reuse Strategy=with reuse (APC)2026.06 | 302 | |
| vLLMHardware=Jetson AGX Thor (SM110), Prefix length=4096, Reuse condition=with reuse (APC)2026.06 | 332 | |
| vLLMHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=8192, Reuse Strategy=with reuse (APC)2026.06 | 482 | |
| vLLMHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=2048, Reuse Strategy=with reuse (APC)2026.06 | 571 | |
| vLLMHardware=Jetson AGX Thor (SM110), Prefix length=8192, Reuse condition=with reuse (APC)2026.06 | 586 | |
| vLLMHardware=Jetson AGX Thor (SM110), Prefix length=2048, Reuse condition=with reuse (APC)2026.06 | 663 | |
| vLLMHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=16384, Reuse Strategy=with reuse (APC)2026.06 | 839 | |
| FlashRTHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=2048, Reuse Strategy=no reuse (cold)2026.06 | 906 | |
| vLLMHardware=Jetson AGX Thor (SM110), Prefix length=16384, Reuse condition=with reuse (APC)2026.06 | 1,036 | |
| FlashRTHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=4096, Reuse Strategy=no reuse (cold)2026.06 | 1,652 | |
| vLLMHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=2048, Reuse Strategy=no reuse (cold)2026.06 | 2,122 | |
| FlashRTHardware=Jetson AGX Thor (SM110), Prefix length=2048, Reuse condition=no reuse (cold)2026.06 | 2,196 | |
| vLLMHardware=Jetson AGX Thor (SM110), Prefix length=2048, Reuse condition=no reuse (cold)2026.06 | 2,529 | |
| FlashRTHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=8192, Reuse Strategy=no reuse (cold)2026.06 | 3,221 | |
| vLLMHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=4096, Reuse Strategy=no reuse (cold)2026.06 | 4,056 | |
| FlashRTHardware=Jetson AGX Thor (SM110), Prefix length=4096, Reuse condition=no reuse (cold)2026.06 | 4,208 | |
| vLLMHardware=Jetson AGX Thor (SM110), Prefix length=4096, Reuse condition=no reuse (cold)2026.06 | 5,197 | |
| FlashRTHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=16384, Reuse Strategy=no reuse (cold)2026.06 | 6,613 | |
| vLLMHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=8192, Reuse Strategy=no reuse (cold)2026.06 | 7,935 | |
| FlashRTHardware=Jetson AGX Thor (SM110), Prefix length=8192, Reuse condition=no reuse (cold)2026.06 | 8,502 | |
| vLLMHardware=Jetson AGX Thor (SM110), Prefix length=8192, Reuse condition=no reuse (cold)2026.06 | 10,121 | |
| vLLMHardware=NVIDIA DGX Spark (GB10, sm_121), Prefix=16384, Reuse Strategy=no reuse (cold)2026.06 | 16,129 | |
| FlashRTHardware=Jetson AGX Thor (SM110), Prefix length=16384, Reuse condition=no reuse (cold)2026.06 | 17,922 | |
| vLLMHardware=Jetson AGX Thor (SM110), Prefix length=16384, Reuse condition=no reuse (cold)2026.06 | 20,748 |