Training Data Extraction on URL PII
2LeakagePME
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PMEModel=GPT-J 6B, Prompt Length=502025.06 | 2 | — | — | 98.21 | |
| PMEModel=GPT-J 6B, Prompt Length=2002025.06 | 2 | — | — | 98.81 | |
| GRACEModel=GPT-J 6B, Prompt Length=2002025.06 | 2 | — | — | 98.81 | |
| PMEModel=GPT-J 6B, Prompt Length=1002025.06 | 3 | — | — | 97.97 | |
| GRACEModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 5 | — | — | 93.33 | |
| PMEModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 6 | — | — | 94.34 | |
| PMEModel=GPT Neo 2.7B, Prompt Length=502025.06 | 7 | — | — | 90.54 | |
| DeMemModel=GPT-J 6B, Prompt Length=1002025.06 | 7 | — | — | 95.27 | |
| PMEModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 8 | — | — | 92 | |
| DeMemModel=GPT-J 6B, Prompt Length=2002025.06 | 8 | — | — | 95.24 | |
| GRACEModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 9 | — | — | 91.51 | |
| DeMemModel=GPT-J 6B, Prompt Length=502025.06 | 9 | — | — | 91.96 | |
| PMEModel=GPT Neo 1.3B, Prompt Length=502025.06 | 11 | — | — | 79.25 | |
| MEMITModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 11 | — | — | 85.33 | |
| MEMITModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 13 | — | — | 87.74 | |
| PMEModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 15 | — | — | 79.73 | |
| PMEModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 16 | — | — | 78.67 | |
| MEMITModel=GPT-J 6B, Prompt Length=2002025.06 | 16 | — | — | 90.48 | |
| MEMITModel=GPT-J 6B, Prompt Length=1002025.06 | 23 | — | — | 84.46 | |
| MEMITModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 25 | — | — | 66.22 | |
| MEMITModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 25 | — | — | 75 | |
| MEMITModel=GPT Neo 1.3B, Prompt Length=502025.06 | 30 | — | — | 43.4 | |
| MEMITModel=GPT Neo 2.7B, Prompt Length=502025.06 | 35 | — | — | 52.7 | |
| MEMITModel=GPT-J 6B, Prompt Length=502025.06 | 39 | — | — | 65.18 | |
| DeMemModel=GPT Neo 1.3B, Prompt Length=502025.06 | 40 | — | — | 24.53 | |
| Pre EditModel=GPT Neo 1.3B, Prompt Length=502025.06 | 53 | 2,002 | 2.6 | — | |
| GRACEModel=GPT Neo 1.3B, Prompt Length=502025.06 | 53 | — | — | 0 | |
| DeMemModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 56 | — | — | 24.32 | |
| DeMemModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 56 | — | — | 25.33 | |
| DeMemModel=GPT Neo 2.7B, Prompt Length=502025.06 | 56 | — | — | 24.32 | |
| DeMemModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 61 | — | — | 42.45 | |
| DeMemModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 63 | — | — | 37 | |
| GRACEModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 70 | — | — | 5.41 | |
| Pre EditModel=GPT Neo 1.3B, Prompt Length=1002025.06 | 74 | 2,012 | 3.7 | — | |
| Pre EditModel=GPT Neo 2.7B, Prompt Length=502025.06 | 74 | 2,088 | 3.5 | — | |
| GRACEModel=GPT Neo 2.7B, Prompt Length=502025.06 | 74 | — | — | 0 | |
| Pre EditModel=GPT Neo 1.3B, Prompt Length=2002025.06 | 75 | 2,017 | 3.7 | — | |
| GRACEModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 93 | — | — | 7 | |
| Pre EditModel=GPT Neo 2.7B, Prompt Length=1002025.06 | 100 | 2,124 | 4.7 | — | |
| Pre EditModel=GPT Neo 2.7B, Prompt Length=2002025.06 | 106 | 2,131 | 5 | — | |
| Pre EditModel=GPT-J 6B, Prompt Length=502025.06 | 112 | 2,288 | 4.9 | — | |
| GRACEModel=GPT-J 6B, Prompt Length=502025.06 | 112 | — | — | 0 | |
| GRACEModel=GPT-J 6B, Prompt Length=1002025.06 | 139 | — | — | 6.08 | |
| Pre EditModel=GPT-J 6B, Prompt Length=1002025.06 | 148 | 2,327 | 6.4 | — | |
| Pre EditModel=GPT-J 6B, Prompt Length=2002025.06 | 168 | 2,333 | 7.2 | — |