# HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption

Source: [HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption](<https://www.sota2.com/research/paper/68fa73b51e209131cea5e09b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:55:12.649Z

Authors: Seewoo Lee, Garam Lee, Jung Woo Kim, Junbum Shin, Mun-Kyu Lee

Published: 2024-03-21T03:47:26.000Z

## Abstract

Transfer learning is a de facto standard method for efficiently training machine learning models for data-scarce problems by adding and fine-tuning new classification layers to a model pre-trained on large datasets. Although numerous previous studies proposed to use homomorphic encryption to resolve the data privacy issue in transfer learning in the machine learning as a service setting, most of them only focused on encrypted inference. In this study, we present HETAL, an efficient Homomorphic Encryption based Transfer Learning algorithm, that protects the client's privacy in training tasks by encrypting the client data using the CKKS homomorphic encryption scheme. HETAL is the first practical scheme that strictly provides encrypted training, adopting validation-based early stopping and achieving the accuracy of nonencrypted training. We propose an efficient encrypted matrix multiplication algorithm, which is 1.8 to 323 times faster than prior methods, and a highly precise softmax approximation algorithm with increased coverage. The experimental results for five well-known benchmark datasets show total training times of 567-3442 seconds, which is less than an hour.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Sparse Matrix-Vector multiplication on ARC 130](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-arc-130.md>): Sparse Matrix-Vector Multiplication; ARC 130; Memory Consumption (MB): 8.06.

- [Sparse Matrix-Vector multiplication on M80PI n1](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-m80pi-n1.md>): Sparse Matrix-Vector Multiplication; M80PI n1; Memory (MB): 2521.64.

- [Sparse Matrix-Vector Multiplication on SuiteSparse arc130 (test)](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-suitesparse-arc130-test.md>): Sparse Matrix-Vector Multiplication; SuiteSparse arc130 (test); Time (s): 8.33.

- [Sparse Matrix-Vector Multiplication on SuiteSparse stat96 v5 (test)](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-suitesparse-stat96-v5-test.md>): Sparse Matrix-Vector Multiplication; SuiteSparse stat96 v5 (test); Time (s): 40436.16.

- [Sparse Matrix-Vector Multiplication on SuiteSparse M80PI\_n1 (test)](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-suitesparse-m80pi_n1-test.md>): Sparse Matrix-Vector Multiplication; SuiteSparse M80PI\_n1 (test); Time (s): 3659.79.

- [Sparse Matrix-Vector Multiplication on SuiteSparse mycielskian13 (test)](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-suitesparse-mycielskian13-test.md>): Sparse Matrix-Vector Multiplication; SuiteSparse mycielskian13 (test); Execution Time (s): 8497.93.

- [Sparse Matrix-Vector Multiplication on SuiteSparse nemeth24 (test)](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-suitesparse-nemeth24-test.md>): Sparse Matrix-Vector Multiplication; SuiteSparse nemeth24 (test); Execution Time (s): 20502.51.

- [Sparse Matrix-Vector Multiplication on SuiteSparse as-caida (test)](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-suitesparse-as-caida-test.md>): Sparse Matrix-Vector Multiplication; SuiteSparse as-caida (test); Execution Time (s): 159955.17.

- [Memory Evaluation for Sparse Matrix-Vector multiplication on stat96 v5](<https://www.sota2.com/research/sota/memory-evaluation-for-sparse-matrix-vector-multiplication-on-stat96-v5.md>): Sparse Matrix-Vector Multiplication; stat96 v5; Memory (MB): 28938.01.

- [Sparse Matrix-Vector multiplication on mycielskian13](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-mycielskian13.md>): Sparse Matrix-Vector Multiplication; mycielskian13; Memory (MB): 6037.39.

- [Memory Evaluation for Sparse Matrix-Vector multiplication on nemeth 24](<https://www.sota2.com/research/sota/memory-evaluation-for-sparse-matrix-vector-multiplication-on-nemeth-24.md>): Sparse Matrix-Vector Multiplication; nemeth 24; Memory (MB): 14914.1.

- [Sparse Matrix-Vector multiplication on msc10848](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-msc10848.md>): Sparse Matrix-Vector Multiplication; msc10848; Memory (MB): 15028.

- [Sparse Matrix-Vector multiplication on as-caida](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-as-caida.md>): Sparse Matrix-Vector Multiplication; as-caida; Memory Consumption (MB): 114161.6.

- [Sparse Matrix-Vector multiplication on ship 001 (Memory Metrics)](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-ship-001-memory-metrics.md>): Sparse Matrix-Vector Multiplication; ship 001; Memory (MB): 155304.24.

- [Sparse Matrix-Vector multiplication on cca](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-cca.md>): Sparse Matrix-Vector Multiplication; cca; Memory (MB): 307107.84.

- [Sparse Matrix-Vector multiplication on p2p-Gnutella 31](<https://www.sota2.com/research/sota/sparse-matrix-vector-multiplication-on-p2p-gnutella-31.md>): Sparse Matrix-Vector Multiplication; p2p-Gnutella 31; Memory (MB): 497880.24.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2403.14111>)

[PDF](<https://arxiv.org/pdf/2403.14111.pdf>)
