Latency and Throughput on QQP (Paraphrase Identification)
22.69Latency (ms)flashsvd15
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| flashsvd15Backend=flashsvd15, Precision=bf16, Sequence Length=512, Batch Size=32, Input Type=synthetic, Model=SVD-compressed BERT2026.05 | 22.69 | 1,410.6 | 345.4 | |
| sdpaBackend=sdpa, Precision=bf16, Sequence Length=512, Batch Size=32, Input Type=synthetic, Model=SVD-compressed BERT2026.05 | 25.48 | 1,255.8 | 605.2 | |
| flashsvdBackend=flashsvd, Precision=bf16, Sequence Length=512, Batch Size=32, Input Type=synthetic, Model=SVD-compressed BERT2026.05 | 44.67 | 716.3 | 341.4 | |
| naiveBackend=naive, Precision=bf16, Sequence Length=512, Batch Size=32, Input Type=synthetic, Model=SVD-compressed BERT2026.05 | 51.37 | 622.9 | 989.2 |