Private Transformer Inference Efficiency on Transformer Block BERT-base settings
3.05Communication Cost (MB)Nimbus
Evaluation results
| Method | Links | ||
|---|---|---|---|
| NimbusLayer=Linearo2024.11 | 3.05 | 1 | |
| NimbusLayer=Linearh22024.11 | 3.05 | 1 | |
| BumbleBeeLayer=Linearo2024.11 | 6.71 | 2 | |
| NimbusLayer=Linearqkv2024.11 | 10.35 | 1 | |
| BumbleBeeLayer=Linearqkv2024.11 | 14.47 | 2 | |
| NimbusLayer=Linearh12024.11 | 15.52 | 1 | |
| BumbleBeeLayer=Linearh22024.11 | 15.71 | 2 | |
| BumbleBeeLayer=Linearh12024.11 | 18.35 | 2 | |
| IronLayer=Linearo2024.11 | 40.2 | 2 | |
| NimbusLayer=GELU2024.11 | 53.22 | 24 | |
| IronLayer=Linearqkv2024.11 | 74.64 | 2 | |
| IronLayer=Linearh22024.11 | 78.37 | 2 | |
| IronLayer=Linearh12024.11 | 84.46 | 2 | |
| NimbusLayer=Softmax2024.11 | 115.35 | 60 | |
| BumbleBeeLayer=GELU2024.11 | 185.13 | 46 | |
| IronLayer=GELU2024.11 | 283.89 | 65 | |
| BumbleBeeLayer=Softmax2024.11 | 354.26 | 70 | |
| IronLayer=Softmax2024.11 | 689.45 | 110 |