List your company
ResearchFormerly WizWand

Latency (ms) on Attention layer forward pass on Synthetic Attention Layer Input

145.1Latency (ms)Query Split (Flash-TPSP)
139.716176.058212.4248.742Apr 27, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Query Split (Flash-TPSP)Data (B x S)=4 × 8192, Embedding Dimension=40962026.04145.1
Query Split (Flash-TPSP)Data (B x S)=64 × 1024, Embedding Dimension=40962026.04164.2
Row Parallel (Flash-TPSP)Data (B x S)=64 × 1024, Embedding Dimension=40962026.04170.7
Row Parallel (Flash-TPSP)Data (B x S)=4 × 8192, Embedding Dimension=40962026.04174.6
SlicingData (B x S)=64 × 1024, Embedding Dimension=40962026.04179.6
Query Split (Flash-TPSP)Data (B x S)=32 × 2048, Embedding Dimension=40962026.04181.2
BaselineData (B x S)=4 × 8192, Embedding Dimension=40962026.04195.6
BaselineData (B x S)=64 × 1024, Embedding Dimension=40962026.04196.2
Row Parallel (Flash-TPSP)Data (B x S)=32 × 2048, Embedding Dimension=40962026.04200.7
SlicingData (B x S)=32 × 2048, Embedding Dimension=40962026.04202.2
Query Split (Flash-TPSP)Data (B x S)=16 × 4096, Embedding Dimension=40962026.04215.8
BaselineData (B x S)=32 × 2048, Embedding Dimension=40962026.04222.8
Row Parallel (Flash-TPSP)Data (B x S)=16 × 4096, Embedding Dimension=40962026.04230.9
SlicingData (B x S)=16 × 4096, Embedding Dimension=40962026.04246.1
BaselineData (B x S)=16 × 4096, Embedding Dimension=40962026.04270.1
SlicingData (B x S)=4 × 8192, Embedding Dimension=40962026.04279.7