Secure Inference on Transformer Models
42.6Online Time (ms)FuseFSS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| FuseFSSModel=BERT-tiny, Sequence length=128, Batch size=12026.06 | 42.6 | 33.3 | 0.018 | 0.06 | 0.268 | |
| SigmaModel=BERT-tiny, Sequence length=128, Batch size=12026.06 | 63.9 | — | 0.021 | 0.07 | 0.35 | |
| FuseFSSModel=GPT-2, Sequence length=128, Batch size=12026.06 | 1,072.7 | 24.7 | 0.777 | 0.99 | 11.92 | |
| FuseFSSModel=BERT-base, Sequence length=128, Batch size=12026.06 | 1,149.5 | 28.8 | 0.891 | 1.08 | 13.678 | |
| SigmaModel=GPT-2, Sequence length=128, Batch size=12026.06 | 1,423.9 | — | 0.885 | 1.2 | 15.346 | |
| SigmaModel=BERT-base, Sequence length=128, Batch size=12026.06 | 1,613.8 | — | 1.062 | 1.32 | 18.076 | |
| FuseFSSModel=BERT-large, Sequence length=128, Batch size=12026.06 | 2,997.9 | 25.7 | 2.376 | 2.48 | 37.075 | |
| SigmaModel=BERT-large, Sequence length=128, Batch size=12026.06 | 4,034.5 | — | 2.833 | 3.21 | 48.799 | |
| FuseFSSModel=GPT-Neo, Sequence length=128, Batch size=12026.06 | 5,115.8 | 19.1 | 3.917 | 4.35 | 65.729 | |
| SigmaModel=GPT-Neo, Sequence length=128, Batch size=12026.06 | 6,326.2 | — | 4.326 | 5.42 | 81.805 |