Language Modeling on Yelp
26.84PPLGPT-2 Large + DP-SGD (teacher)
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-2 Large + DP-SGD (teacher)ε=1.02026.04 | 26.84 | |
| DP-OPDε=2.02026.04 | 41.68 | |
| DistilGPT-2 + DISTILDP (DP synthetic text)ε=2.02026.04 | 44.15 | |
| DistilGPT-2 + DPKD (off-policy DP KD)ε=2.02026.04 | 46.16 | |
| DistilGPT-2 + DP-SGD (student-only)ε=2.02026.04 | 48.12 |