Membership Inference Attack on GitHub
0.876AUCSAMA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SAMA2026.01 | 0.876 | 64.7 | 25.9 | 7.5 | — | |
| SAMAModel=Dream-v0-Base-7B2026.01 | 0.806 | 49.8 | 16.8 | 4.2 | — | |
| AutoMIATarget model=Pythia 1.4B2026.03 | 0.75 | — | 13.4 | — | 35.1 | |
| Ratio2026.01 | 0.743 | 35.5 | 8.1 | 1.7 | — | |
| OpenEvolveTarget model=Pythia 1.4B2026.03 | 0.719 | — | 5.2 | — | 22.4 | |
| Hallinan et al.Target model=Pythia 1.4B2026.03 | 0.664 | — | 2.2 | — | 20.9 | |
| BoWsModel=Dream-v0-Base-7B2026.01 | 0.656 | 30.6 | 15.4 | 5.9 | — | |
| BoWs2026.01 | 0.656 | 30.6 | 15.4 | 5.9 | — | |
| AutoMIATarget model=OPT 7B2026.03 | 0.653 | — | 14.2 | — | 21.6 | |
| Hallinan et al.Target model=OPT 7B2026.03 | 0.62 | — | 11.2 | — | 15.7 | |
| OpenEvolveTarget model=OPT 7B2026.03 | 0.609 | — | 10.4 | — | 14.2 | |
| SecMI2026.01 | 0.604 | 19 | 4.4 | 1.9 | — | |
| Lowercase2026.01 | 0.579 | 17.8 | 4.4 | 0.8 | — | |
| PIA2026.01 | 0.571 | 13.1 | 1.2 | 0.1 | — | |
| ReCall2026.01 | 0.562 | 18.7 | 3.7 | 0.7 | — | |
| ZLIB2026.01 | 0.561 | 20.5 | 4.5 | 0.7 | — | |
| ZLIBModel=Dream-v0-Base-7B2026.01 | 0.558 | 16.8 | 2.8 | 0.4 | — | |
| Loss2026.01 | 0.551 | 16.1 | 3.6 | 0.7 | — | |
| RatioModel=Dream-v0-Base-7B2026.01 | 0.549 | 16.2 | 2.9 | 0.4 | — | |
| CON-Recall2026.01 | 0.549 | 16.8 | 2.7 | 0.5 | — | |
| SecMIModel=Dream-v0-Base-7B2026.01 | 0.537 | 14.9 | 2.5 | 0.4 | — | |
| ReCallModel=Dream-v0-Base-7B2026.01 | 0.531 | 14.2 | 2.3 | 0.3 | — | |
| Min-K%2026.01 | 0.53 | 17.1 | 3.9 | 0.7 | — | |
| LossModel=Dream-v0-Base-7B2026.01 | 0.524 | 12.8 | 1.8 | 0.2 | — | |
| CON-RecallModel=Dream-v0-Base-7B2026.01 | 0.523 | 13.4 | 1.9 | 0.2 | — | |
| Min-K%Model=Dream-v0-Base-7B2026.01 | 0.515 | 11.9 | 2.1 | 0.3 | — | |
| PIAModel=Dream-v0-Base-7B2026.01 | 0.509 | 11.3 | 1.3 | 0.1 | — | |
| LowercaseModel=Dream-v0-Base-7B2026.01 | 0.498 | 10.4 | 1.4 | 0.1 | — | |
| Min-K%++2026.01 | 0.496 | 11.7 | 1.6 | 0.3 | — | |
| Min-K%++Model=Dream-v0-Base-7B2026.01 | 0.483 | 9.4 | 1.1 | 0.1 | — | |
| NeighborModel=Dream-v0-Base-7B2026.01 | 0.479 | 8.2 | 0.7 | 0 | — | |
| Neighbor2026.01 | 0.478 | 7.2 | 0.8 | 0 | — |