ResearchBenchmarksLLM Inference Latency on Multi-node InfiniBand Cluster (2x8-GPU Servers) (test)LLM Inference Latency on Multi-node InfiniBand Cluster (2x8-GPU Servers) (test)Follow0.939Time To First Token (TTFT) Mean (s)FoE0.821241.616122.4113.20588May 7, 2026Evaluation resultsMethodMethodLinksTime To First Token (TTFT) Mean (s)Time To First Token (TTFT) p50 (s)Time To First Token (TTFT) p99 (s)Time Between Tokens (TBT) Mean (s)Time Between Tokens (TBT) p50 (s)Time Between Tokens (TBT) p99 (s)End-to-End Latency (E2E) Mean (s)End-to-End Latency (E2E) p50 (s)End-to-End Latency (E2E) p99 (s)FoEScale=0.75Scale=0.752026.050.9390.9081.6790.4490.4460.495115.4115.2127.3FoEScale=0.5Scale=0.52026.051.1281.0382.5750.5430.5570.601139.5143.5154.4MoEScale=0.75Scale=0.752026.053.4033.2616.2180.8760.8951.12226.7232.1288.7MoEScale=0.5Scale=0.52026.053.8833.7087.0660.7360.7390.94191.5192.5242.6