Throughput Scalability on General Inference Workload
50,000RPSSwiftEmbed
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SwiftEmbedBatch size=1, Scaling=Linear2025.10 | 50,000 | 10,000 | |
| FastText++Batch size=1, Scaling=Linear2025.10 | 15,000 | 2,000 | |
| FastText++Batch size=10, Scaling=Linear2025.10 | 12,000 | 2,000 | |
| SwiftEmbedBatch size=10, Scaling=Linear2025.10 | 10,000 | 10,000 | |
| TensorRT-BERTBatch size=1, Scaling=Sub-quadratic2025.10 | 8,500 | 1,000 | |
| FastText++Batch size=100, Scaling=Linear2025.10 | 8,000 | 2,000 | |
| DistilBERTBatch size=1, Scaling=Quadratic2025.10 | 4,200 | 800 | |
| TensorRT-BERTBatch size=10, Scaling=Sub-quadratic2025.10 | 4,000 | 1,000 | |
| Sentence-BERTBatch size=1, Scaling=Quadratic2025.10 | 2,500 | 500 | |
| DistilBERTBatch size=10, Scaling=Quadratic2025.10 | 2,100 | 800 | |
| SwiftEmbedBatch size=100, Scaling=Linear2025.10 | 2,000 | 10,000 | |
| Sentence-BERTBatch size=10, Scaling=Quadratic2025.10 | 1,200 | 500 | |
| TensorRT-BERTBatch size=100, Scaling=Sub-quadratic2025.10 | 850 | 1,000 | |
| DistilBERTBatch size=100, Scaling=Quadratic2025.10 | 420 | 800 | |
| Sentence-BERTBatch size=100, Scaling=Quadratic2025.10 | 250 | 500 |