Training Data Extraction on Email PII
0LeakagePME
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PMEModel=GPT Neo 1.3B, Prompt Length=502025.06 | 0 | — | — | 100 | |
| MEMITModel=GPT Neo 1.3B, Prompt Length=502025.06 | 0 | — | — | 100 | |
| PMEModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 0 | — | — | 100 | |
| PMEModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 0 | — | — | 100 | |
| GRACEModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 0 | — | — | 100 | |
| PMEModel=GPT Neo 2.7B, Prompt Length=502025.06 | 0 | — | — | 100 | |
| MEMITModel=GPT Neo 2.7B, Prompt Length=502025.06 | 0 | — | — | 100 | |
| PMEModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 0 | — | — | 100 | |
| PMEModel=GPT-J 6B, Prompt Length=2002025.06 | 0 | — | — | 100 | |
| MEMITModel=GPT-J 6B, Prompt Length=2002025.06 | 0 | — | — | 100 | |
| MEMITModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 1 | — | — | 99.44 | |
| MEMITModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 1 | — | — | 99.59 | |
| PMEModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 1 | — | — | 99.65 | |
| MEMITModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 1 | — | — | 99.65 | |
| PMEModel=GPT-J 6B, Prompt Length=502025.06 | 1 | — | — | 99.72 | |
| MEMITModel=GPT-J 6B, Prompt Length=502025.06 | 1 | — | — | 99.72 | |
| PMEModel=GPT-J 6B, Prompt Length=1002025.06 | 1 | — | — | 99.79 | |
| MEMITModel=GPT-J 6B, Prompt Length=1002025.06 | 1 | — | — | 99.79 | |
| MEMITModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 2 | — | — | 98.65 | |
| GRACEModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 2 | — | — | 99.3 | |
| GRACEModel=GPT-J 6B, Prompt Length=2002025.06 | 7 | — | — | 98.7 | |
| DeMemModel=GPT-J 6B, Prompt Length=502025.06 | 25 | — | — | 92.92 | |
| DeMemModel=GPT-J 6B, Prompt Length=1002025.06 | 33 | — | — | 93.07 | |
| DeMemModel=GPT-J 6B, Prompt Length=2002025.06 | 33 | — | — | 93.85 | |
| DeMemModel=GPT Neo 1.3B, Prompt Length=502025.06 | 59 | — | — | 38.54 | |
| DeMemModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 77 | — | — | 47.97 | |
| DeMemModel=GPT Neo 2.7B, Prompt Length=502025.06 | 77 | — | — | 56.25 | |
| DeMemModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 88 | — | — | 50.84 | |
| GRACEModel=GPT Neo 1.3B, Prompt Length=502025.06 | 89 | — | — | 7.29 | |
| Pre EditModel=GPT Neo 1.3B, Prompt Length=502025.06 | 96 | 2,789 | 3.4 | — | |
| DeMemModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 96 | — | — | 60.98 | |
| DeMemModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 102 | — | — | 64.34 | |
| GRACEModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 136 | — | — | 8.11 | |
| Pre EditModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 148 | 2,876 | 5.1 | — | |
| GRACEModel=GPT Neo 2.7B, Prompt Length=502025.06 | 156 | — | — | 11.36 | |
| Pre EditModel=GPT Neo 2.7B, Prompt Length=502025.06 | 176 | 2,884 | 6.1 | — | |
| Pre EditModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 179 | 2,899 | 6.2 | — | |
| GRACEModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 207 | — | — | 15.85 | |
| Pre EditModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 246 | 2,973 | 8.3 | — | |
| Pre EditModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 286 | 2,973 | 9.6 | — | |
| GRACEModel=GPT-J 6B, Prompt Length=502025.06 | 313 | — | — | 11.33 | |
| Pre EditModel=GPT-J 6B, Prompt Length=502025.06 | 353 | 2,827 | 12.5 | — | |
| GRACEModel=GPT-J 6B, Prompt Length=1002025.06 | 386 | — | — | 18.91 | |
| Pre EditModel=GPT-J 6B, Prompt Length=1002025.06 | 476 | 2,932 | 16.2 | — | |
| Pre EditModel=GPT-J 6B, Prompt Length=2002025.06 | 537 | 2,951 | 18.2 | — |