Auditing Data Removal on MNIST Memorized (train folds M1-M5)
1.07pksLiu et al. method
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Liu et al. methodk (calibration set quality)=50, Query dataset fold=M1, Target model=3-layer MLP2021.09 | 1.07 | — | |
| Liu et al. methodk (calibration set quality)=50, Query dataset fold=M3, Target model=3-layer MLP2021.09 | 1.06 | — | |
| Liu et al. methodk (calibration set quality)=50, Query dataset fold=M4, Target model=3-layer MLP2021.09 | 1.05 | — | |
| Liu et al. methodk (calibration set quality)=50, Query dataset fold=M5, Target model=3-layer MLP2021.09 | 1.05 | — | |
| Liu et al. methodk (calibration set quality)=50, Query dataset fold=M2, Target model=3-layer MLP2021.09 | 1.04 | — | |
| Liu et al. methodk (calibration set quality)=60, Query dataset fold=M1, Target model=3-layer MLP2021.09 | 1.02 | — | |
| Liu et al. methodk (calibration set quality)=60, Query dataset fold=M3, Target model=3-layer MLP2021.09 | 1.01 | — | |
| Liu et al. methodk (calibration set quality)=60, Query dataset fold=M4, Target model=3-layer MLP2021.09 | 1 | — | |
| Ensembled Membership Auditing (EMA)k (calibration set quality)=100, Query dataset fold=M1-M5, Target model=3-layer MLP2021.09 | 1 | — | |
| Ensembled Membership Auditing (EMA)k (calibration set quality)=90, Query dataset fold=M1-M5, Target model=3-layer MLP2021.09 | 1 | — | |
| Ensembled Membership Auditing (EMA)k (calibration set quality)=80, Query dataset fold=M1-M5, Target model=3-layer MLP2021.09 | 1 | — | |
| Ensembled Membership Auditing (EMA)k (calibration set quality)=70, Query dataset fold=M1-M5, Target model=3-layer MLP2021.09 | 1 | — | |
| Ensembled Membership Auditing (EMA)k (calibration set quality)=60, Query dataset fold=M1-M5, Target model=3-layer MLP2021.09 | 1 | — | |
| Ensembled Membership Auditing (EMA)k (calibration set quality)=50, Query dataset fold=M1-M5, Target model=3-layer MLP2021.09 | 1 | — | |
| Liu et al. methodk (calibration set quality)=60, Query dataset fold=M2, Target model=3-layer MLP2021.09 | 0.99 | — | |
| Liu et al. methodk (calibration set quality)=60, Query dataset fold=M5, Target model=3-layer MLP2021.09 | 0.99 | — | |
| Liu et al. methodk (calibration set quality)=80, Query dataset fold=M1, Target model=3-layer MLP2021.09 | 0.98 | — | |
| Liu et al. methodk (calibration set quality)=70, Query dataset fold=M1, Target model=3-layer MLP2021.09 | 0.98 | — | |
| Liu et al. methodk (calibration set quality)=80, Query dataset fold=M3, Target model=3-layer MLP2021.09 | 0.97 | — | |
| Liu et al. methodk (calibration set quality)=70, Query dataset fold=M3, Target model=3-layer MLP2021.09 | 0.97 | — | |
| Liu et al. methodk (calibration set quality)=90, Query dataset fold=M1, Target model=3-layer MLP2021.09 | 0.96 | — | |
| Liu et al. methodk (calibration set quality)=90, Query dataset fold=M3, Target model=3-layer MLP2021.09 | 0.96 | — | |
| Liu et al. methodk (calibration set quality)=80, Query dataset fold=M5, Target model=3-layer MLP2021.09 | 0.96 | — | |
| Liu et al. methodk (calibration set quality)=70, Query dataset fold=M2, Target model=3-layer MLP2021.09 | 0.96 | — | |
| Liu et al. methodk (calibration set quality)=70, Query dataset fold=M4, Target model=3-layer MLP2021.09 | 0.96 | — | |
| Liu et al. methodk (calibration set quality)=70, Query dataset fold=M5, Target model=3-layer MLP2021.09 | 0.96 | — | |
| Liu et al. methodk (calibration set quality)=90, Query dataset fold=M2, Target model=3-layer MLP2021.09 | 0.95 | — | |
| Liu et al. methodk (calibration set quality)=90, Query dataset fold=M4, Target model=3-layer MLP2021.09 | 0.95 | — | |
| Liu et al. methodk (calibration set quality)=90, Query dataset fold=M5, Target model=3-layer MLP2021.09 | 0.95 | — | |
| Liu et al. methodk (calibration set quality)=80, Query dataset fold=M2, Target model=3-layer MLP2021.09 | 0.95 | — | |
| Liu et al. methodk (calibration set quality)=80, Query dataset fold=M4, Target model=3-layer MLP2021.09 | 0.95 | — | |
| Liu et al. methodk (calibration set quality)=100, Query dataset fold=M1, Target model=3-layer MLP2021.09 | 0.91 | — | |
| Liu et al. methodk (calibration set quality)=100, Query dataset fold=M2, Target model=3-layer MLP2021.09 | 0.9 | — | |
| Liu et al. methodk (calibration set quality)=100, Query dataset fold=M3, Target model=3-layer MLP2021.09 | 0.9 | — | |
| Liu et al. methodk (calibration set quality)=100, Query dataset fold=M5, Target model=3-layer MLP2021.09 | 0.9 | — | |
| Liu et al. methodk (calibration set quality)=100, Query dataset fold=M4, Target model=3-layer MLP2021.09 | 0.89 | — | |
| EMACalibration set quality (k)=100, Query dataset fold (M1-M5)=M1, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=100, Query dataset fold (M1-M5)=M2, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=100, Query dataset fold (M1-M5)=M3, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=100, Query dataset fold (M1-M5)=M4, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=100, Query dataset fold (M1-M5)=M5, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=90, Query dataset fold (M1-M5)=M1, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=90, Query dataset fold (M1-M5)=M2, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=90, Query dataset fold (M1-M5)=M3, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=90, Query dataset fold (M1-M5)=M4, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=90, Query dataset fold (M1-M5)=M5, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=80, Query dataset fold (M1-M5)=M1, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=80, Query dataset fold (M1-M5)=M2, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=80, Query dataset fold (M1-M5)=M3, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=80, Query dataset fold (M1-M5)=M4, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=80, Query dataset fold (M1-M5)=M5, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=70, Query dataset fold (M1-M5)=M1, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=70, Query dataset fold (M1-M5)=M2, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=70, Query dataset fold (M1-M5)=M3, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=70, Query dataset fold (M1-M5)=M4, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=70, Query dataset fold (M1-M5)=M5, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=60, Query dataset fold (M1-M5)=M1, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=60, Query dataset fold (M1-M5)=M2, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=60, Query dataset fold (M1-M5)=M3, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=60, Query dataset fold (M1-M5)=M4, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=60, Query dataset fold (M1-M5)=M5, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=50, Query dataset fold (M1-M5)=M1, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=50, Query dataset fold (M1-M5)=M2, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=50, Query dataset fold (M1-M5)=M3, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=50, Query dataset fold (M1-M5)=M4, Evaluation protocol (t-test)=t-test2021.09 | — | 1 | |
| EMACalibration set quality (k)=50, Query dataset fold (M1-M5)=M5, Evaluation protocol (t-test)=t-test2021.09 | — | 1 |