ResearchBenchmarksModel Memory Footprint Analysis on T5 138M (Private Text Generation)Follow100Memory FractionCrypTen22.800842.842962.88582.9271May 25, 2023Evaluation ResultsMethodMethodLinksMemory FractionEmbed Memory (GB)Linear Layer Memory (GB)Softmax Layer Memory (GB)Total Memory Footprint (GB)CrypTen2023.05100————THE-X2023.0597.18————MPCformersoftmax_approximation=...softmax_approximation=sm2relu2023.0595.87————MPCformersoftmax_approximation=...softmax_approximation=sm2quad2023.0595.24————MERGEmode=only MMmode=only MM2023.0567.79————MERGEmode=only ERmode=only ER2023.0534.81————MERGEmode=ER+MMmode=ER+MM2023.0525.77————