Prefill Latency on Llama-3-8B 10 Mbps
1.563Prefill Latency (s)ASTRA
Evaluation results
| Method | Links | |
|---|---|---|
| ASTRAG=12025.05 | 1.563 | |
| ASTRAG=162025.05 | 1.661 | |
| ASTRAG=322025.05 | 1.94 | |
| Llama-3-8B (Single-device)Device=Single-device2025.05 | 4.578 | |
| BPStrategy=Block Parallelism, Nb=42025.05 | 4.642 | |
| BPStrategy=Block Parallelism, Nb=82025.05 | 8.011 | |
| SPStrategy=Sequence Parallelism2025.05 | 28.256 | |
| TPStrategy=Tensor Parallelism2025.05 | 430.952 |