Natural Language Inference on MNLI (Latency, Throughput, Memory)
22.52Latency (ms)flashsvd15
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| flashsvd15Backend=flashsvd15, Precision=bf16, Sequence Length=512, Batch Size=32, Input Type=synthetic, Model=SVD-compressed BERT2026.05 | 22.52 | 1,421 | 343.9 | |
| sdpaBackend=sdpa, Precision=bf16, Sequence Length=512, Batch Size=32, Input Type=synthetic, Model=SVD-compressed BERT2026.05 | 25.33 | 1,263.1 | 605.2 | |
| flashsvdBackend=flashsvd, Precision=bf16, Sequence Length=512, Batch Size=32, Input Type=synthetic, Model=SVD-compressed BERT2026.05 | 44.4 | 720.8 | 341.4 | |
| naiveBackend=naive, Precision=bf16, Sequence Length=512, Batch Size=32, Input Type=synthetic, Model=SVD-compressed BERT2026.05 | 51.34 | 623.3 | 989.2 |