Training Data Extraction on Phone PII
0Leak CountPME
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PMEModel=GPT Neo 1.3B, Prompt Length=502025.06 | 0 | — | — | 100 | |
| GRACEModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 0 | — | — | 100 | |
| PMEModel=GPT Neo 2.7B, Prompt Length=502025.06 | 0 | — | — | 100 | |
| PMEModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 0 | — | — | 100 | |
| GRACEModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 0 | — | — | 100 | |
| DeMemModel=GPT-J 6B, Prompt Length=502025.06 | 0 | — | — | 100 | |
| DeMemModel=GPT-J 6B, Prompt Length=1002025.06 | 0 | — | — | 100 | |
| GRACEModel=GPT-J 6B, Prompt Length=2002025.06 | 0 | — | — | 100 | |
| PMEModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 1 | — | — | 96.3 | |
| PMEModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 1 | — | — | 97.06 | |
| PMEModel=GPT-J 6B, Prompt Length=502025.06 | 1 | — | — | 98.99 | |
| MEMITModel=GPT-J 6B, Prompt Length=502025.06 | 1 | — | — | 98.99 | |
| MEMITModel=GPT-J 6B, Prompt Length=2002025.06 | 1 | — | — | 99.38 | |
| MEMITModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 2 | — | — | 94.12 | |
| PMEModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 2 | — | — | 97.3 | |
| MEMITModel=GPT-J 6B, Prompt Length=1002025.06 | 2 | — | — | 98.4 | |
| MEMITModel=GPT Neo 1.3B, Prompt Length=502025.06 | 3 | — | — | 81.25 | |
| MEMITModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 3 | — | — | 88.89 | |
| MEMITModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 3 | — | — | 95.95 | |
| PMEModel=GPT-J 6B, Prompt Length=1002025.06 | 3 | — | — | 97.6 | |
| DeMemModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 4 | — | — | 85.19 | |
| PMEModel=GPT-J 6B, Prompt Length=2002025.06 | 5 | — | — | 96.89 | |
| DeMemModel=GPT-J 6B, Prompt Length=2002025.06 | 5 | — | — | 96.89 | |
| DeMemModel=GPT Neo 1.3B, Prompt Length=502025.06 | 6 | — | — | 62.5 | |
| MEMITModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 6 | — | — | 90 | |
| DeMemModel=GPT Neo 2.7B, Prompt Length=502025.06 | 7 | — | — | 80 | |
| DeMemModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 8 | — | — | 76.47 | |
| MEMITModel=GPT Neo 2.7B, Prompt Length=502025.06 | 8 | — | — | 77.14 | |
| DeMemModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 10 | — | — | 83.33 | |
| DeMemModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 12 | — | — | 83.78 | |
| Pre EditModel=GPT Neo 1.3B, Prompt Length=502025.06 | 16 | 2,790 | 0.6 | — | |
| GRACEModel=GPT Neo 1.3B, Prompt Length=502025.06 | 16 | — | — | 0 | |
| GRACEModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 26 | — | — | 3.7 | |
| Pre EditModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 27 | 2,809 | 1 | — | |
| Pre EditModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 34 | 2,849 | 1.2 | — | |
| Pre EditModel=GPT Neo 2.7B, Prompt Length=502025.06 | 35 | 2,935 | 1.2 | — | |
| GRACEModel=GPT Neo 2.7B, Prompt Length=502025.06 | 35 | — | — | 0 | |
| GRACEModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 57 | — | — | 5 | |
| Pre EditModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 60 | 2,977 | 2 | — | |
| Pre EditModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 74 | 2,983 | 2.5 | — | |
| Pre EditModel=GPT-J 6B, Prompt Length=502025.06 | 99 | 3,132 | 3.2 | — | |
| GRACEModel=GPT-J 6B, Prompt Length=502025.06 | 99 | — | — | 0 | |
| GRACEModel=GPT-J 6B, Prompt Length=1002025.06 | 121 | — | — | 3.2 | |
| Pre EditModel=GPT-J 6B, Prompt Length=1002025.06 | 125 | 3,166 | 3.9 | — | |
| Pre EditModel=GPT-J 6B, Prompt Length=2002025.06 | 161 | 3,240 | 5 | — |