# I Stolenly Swear That I Am Up to (No) Good: Design and Evaluation of Model Stealing Attacks

Source: [I Stolenly Swear That I Am Up to (No) Good: Design and Evaluation of Model Stealing Attacks](<https://www.sota2.com/research/paper/68fa7af31e209131cea85eb5>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:30:46.040Z

Authors: Daryna Oliynyk, Rudolf Mayer, Kathrin Grosse, Andreas Rauber

Published: 2025-08-29T14:16:19.000Z

## Abstract

Model stealing attacks endanger the confidentiality of machine learning models offered as a service. Although these models are kept secret, a malicious party can query a model to label data samples and train their own substitute model, violating intellectual property. While novel attacks in the field are continually being published, their design and evaluations are not standardised, making it challenging to compare prior works and assess progress in the field. This paper is the first to address this gap by providing recommendations for designing and evaluating model stealing attacks. To this end, we study the largest group of attacks that rely on training a substitute model -- those attacking image classification models. We propose the first comprehensive threat model and develop a framework for attack comparison. Further, we analyse attack setups from related works to understand which tasks and models have been studied the most. Based on our findings, we present best practices for attack development before, during, and beyond experiments and derive an extensive list of open research questions regarding the evaluation of model stealing attacks. Our findings and recommendations also transfer to other problem domains, hence establishing the first generic evaluation methodology for model stealing attacks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2508.21654>)

[PDF](<https://arxiv.org/pdf/2508.21654.pdf>)
