Transformer Inference on 12-layer Transformer 1024 tokens
342.74SpeedupASTRA
Evaluation Results
| Method | Links | |
|---|---|---|
| ASTRABandwidth (Mbps)=10, Number of devices=4, Input token length=1024, Baseline model=TP2025.05 | 342.74 | |
| ASTRABandwidth (Mbps)=20, Number of devices=4, Input token length=1024, Baseline model=TP2025.05 | 177.89 | |
| ASTRABandwidth (Mbps)=10, Number of devices=4, Input token length=1024, Baseline model=SP2025.05 | 171.82 | |
| ASTRABandwidth (Mbps)=20, Number of devices=4, Input token length=1024, Baseline model=SP2025.05 | 89.41 | |
| ASTRABandwidth (Mbps)=50, Number of devices=4, Input token length=1024, Baseline model=TP2025.05 | 73.14 | |
| ASTRABandwidth (Mbps)=100, Number of devices=4, Input token length=1024, Baseline model=TP2025.05 | 37.19 | |
| ASTRABandwidth (Mbps)=50, Number of devices=4, Input token length=1024, Baseline model=SP2025.05 | 37.05 | |
| ASTRABandwidth (Mbps)=10, Number of devices=4, Input token length=1024, Baseline model=BP+SP, Nb=12025.05 | 29.37 | |
| ASTRABandwidth (Mbps)=100, Number of devices=4, Input token length=1024, Baseline model=SP2025.05 | 19.08 | |
| ASTRABandwidth (Mbps)=200, Number of devices=4, Input token length=1024, Baseline model=TP2025.05 | 19.02 | |
| ASTRABandwidth (Mbps)=20, Number of devices=4, Input token length=1024, Baseline model=BP+SP, Nb=12025.05 | 15.66 | |
| ASTRABandwidth (Mbps)=10, Number of devices=4, Input token length=1024, Baseline model=BP+AG, Nb=12025.05 | 15.25 | |
| ASTRABandwidth (Mbps)=200, Number of devices=4, Input token length=1024, Baseline model=SP2025.05 | 9.99 | |
| ASTRABandwidth (Mbps)=20, Number of devices=4, Input token length=1024, Baseline model=BP+AG, Nb=12025.05 | 8.41 | |
| ASTRABandwidth (Mbps)=500, Number of devices=4, Input token length=1024, Baseline model=TP2025.05 | 8.05 | |
| ASTRABandwidth (Mbps)=50, Number of devices=4, Input token length=1024, Baseline model=BP+SP, Nb=12025.05 | 6.95 | |
| ASTRABandwidth (Mbps)=500, Number of devices=4, Input token length=1024, Baseline model=SP2025.05 | 4.51 | |
| ASTRABandwidth (Mbps)=50, Number of devices=4, Input token length=1024, Baseline model=BP+AG, Nb=12025.05 | 4.07 | |
| ASTRABandwidth (Mbps)=100, Number of devices=4, Input token length=1024, Baseline model=BP+SP, Nb=12025.05 | 3.96 | |
| ASTRABandwidth (Mbps)=100, Number of devices=4, Input token length=1024, Baseline model=BP+AG, Nb=12025.05 | 2.58 | |
| ASTRABandwidth (Mbps)=200, Number of devices=4, Input token length=1024, Baseline model=BP+SP, Nb=12025.05 | 2.45 | |
| ASTRABandwidth (Mbps)=200, Number of devices=4, Input token length=1024, Baseline model=BP+AG, Nb=12025.05 | 1.83 | |
| ASTRABandwidth (Mbps)=500, Number of devices=4, Input token length=1024, Baseline model=BP+SP, Nb=12025.05 | 1.53 | |
| ASTRABandwidth (Mbps)=500, Number of devices=4, Input token length=1024, Baseline model=BP+AG, Nb=12025.05 | 1.37 |