Computation Cost Measurement on Encrypted Inference 1024 tokens
8.6Computation Cost (s)PowerSoftmax
Evaluation results
| Method | Links | |
|---|---|---|
| PowerSoftmaxModel=Pythia 70M, #Heads=8, #Layers=62024.10 | 8.6 | |
| PowerSoftmaxModel=Pythia 1.4B, #Heads=16, #Layers=242024.10 | 51.8 | |
| Taylor and GoldschmidtModel=Pythia 70M, #Heads=8, #Layers=62024.10 | 83.9 | |
| Square strategyModel=Pythia 70M, #Heads=8, #Layers=62024.10 | 173.4 | |
| Taylor and GoldschmidtModel=Pythia 1.4B, #Heads=16, #Layers=242024.10 | 662.3 | |
| Square strategyModel=Pythia 1.4B, #Heads=16, #Layers=242024.10 | 1,371.6 |