A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Models

About

The development of machine learning (ML) techniques has led to ample opportunities for developers to develop and deploy their own models. Hugging Face serves as an open source platform where developers can share and download other models in an effort to make ML development more collaborative. In order for models to be shared, they first need to be serialized. Certain Python serialization methods are considered unsafe, as they are vulnerable to object injection. This paper investigates the pervasiveness of these unsafe serialization methods across Hugging Face, and demonstrates through an exploitation approach, that models using unsafe serialization methods can be exploited and shared, creating an unsafe environment for ML developers. We investigate to what extent Hugging Face is able to flag repositories and files using unsafe serialization methods, and develop a technique to detect malicious models. Our results show that Hugging Face is home to a wide range of potentially vulnerable models.

Beatrice Casey, Joanna C. S. Santos, Mehdi Mirakhorli · 2024

Related benchmarks

Task	Dataset	Result
Malicious PTM Detection	Malicious PTM Detection Dataset Benign Malicious 2025	Benign Count0	7
Malicious model detection	334 ML models Authors' threat model	FP Count1	6
Malicious model detection	334 ML models Prior work threat model	False Positives Count1	6