Private Inference on GPT2-base (124M)
5.17Embed Inference Time (s)MERGE
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MERGEOptimization Strategy=ER+MM2023.05 | 5.17 | 0.87 | 157.5 | 53.97 | 0 | 0 | 171.38 | 7.75 | |
| MERGEOptimization Strategy=only ER2023.05 | 5.41 | 0.95 | 260.36 | 80 | 477.76 | 124.83 | 834.13 | 1.59 | |
| MPCformerSoftmax Approximation=sm2relu2023.05 | 316.75 | 51.55 | 253.57 | 76.56 | 181.14 | 45.59 | 1,001.41 | 1.33 | |
| MPCformerSoftmax Approximation=sm2quad2023.05 | 318.16 | 50.88 | 253.3 | 75.16 | 152.45 | 37.4 | 972.5 | 1.36 | |
| MERGEOptimization Strategy=only MM2023.05 | 320.84 | 50.92 | 250.98 | 81.57 | 0 | 0 | 747.45 | 1.78 | |
| CrypTen2023.05 | 321.44 | 52.33 | 251.93 | 74.21 | 454.61 | 113.96 | 1,328.26 | 1 | |
| THE-X2023.05 | 329.29 | 58.3 | 258 | 80.21 | 87.71 | 19.28 | 965.79 | 1.37 |