Next Token Prediction on PubMed
42.33Next Token AccuracyDP-FT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DP-FTData Type=Private, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=42026.02 | 42.33 | — | — | — | — | — | — | |
| DP-FTData Type=Private, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=22026.02 | 42.06 | — | — | — | — | — | — | |
| DP-FTData Type=Private, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=12026.02 | 41.64 | — | — | — | — | — | — | |
| DP-FTData Type=Private, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=inf2026.02 | 40.33 | — | — | — | — | — | — | |
| DP-RFTData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=inf2026.02 | 38.31 | — | — | — | — | — | — | |
| DP-RFTData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=42026.02 | 37.37 | — | — | — | — | — | — | |
| DP-RFTData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=22026.02 | 37.19 | — | — | — | — | — | — | |
| DP-RFTData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=12026.02 | 36.75 | — | — | — | — | — | — | |
| RPSGModel=GPT-4o-mini2026.04 | 36.1 | — | — | — | — | 3.11 | 22.5 | |
| AUG-PEModel=GPT-4o-mini2026.04 | 35.8 | — | — | — | — | 2.96 | 19.3 | |
| Aug-PEData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=12026.02 | 34.55 | — | — | — | — | — | — | |
| AUG-PEModel=GPT-3.52026.04 | 34.4 | — | — | — | — | 3.04 | 20.9 | |
| RPSGModel=GPT-3.52026.04 | 34.4 | — | — | — | — | 3.13 | 22.9 | |
| Aug-PEData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=42026.02 | 34.1 | — | — | — | — | — | — | |
| Aug-PEData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=inf2026.02 | 33.91 | — | — | — | — | — | — | |
| Aug-PEData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=22026.02 | 33.57 | — | — | — | — | — | — | |
| QWENData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=inf2026.02 | 33.04 | — | — | — | — | — | — | |
| QWENData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=42026.02 | 33.04 | — | — | — | — | — | — | |
| QWENData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=22026.02 | 33.04 | — | — | — | — | — | — | |
| QWENData Type=Synthetic, Size=2,000, Backbone=GPT-2, Privacy Budget (epsilon)=12026.02 | 33.04 | — | — | — | — | — | — | |
| RPSGModel=Phi-42026.04 | 32.9 | — | — | — | — | 3.34 | 28.1 | |
| AUG-PEModel=Phi-42026.04 | 32.4 | — | — | — | — | 3.41 | 30 | |
| DP-FTData Type=Private, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=inf2026.02 | 20.79 | — | — | — | — | — | — | |
| DP-RFTData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=42026.02 | 17.81 | — | — | — | — | — | — | |
| DP-RFTData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=inf2026.02 | 17.46 | — | — | — | — | — | — | |
| DP-RFTData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=22026.02 | 17.32 | — | — | — | — | — | — | |
| QWENData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=inf2026.02 | 16.88 | — | — | — | — | — | — | |
| QWENData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=42026.02 | 16.88 | — | — | — | — | — | — | |
| QWENData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=22026.02 | 16.88 | — | — | — | — | — | — | |
| QWENData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=12026.02 | 16.88 | — | — | — | — | — | — | |
| DP-RFTData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=12026.02 | 16.79 | — | — | — | — | — | — | |
| Aug-PEData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=42026.02 | 14.41 | — | — | — | — | — | — | |
| Aug-PEData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=22026.02 | 14.16 | — | — | — | — | — | — | |
| Aug-PEData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=inf2026.02 | 13.96 | — | — | — | — | — | — | |
| Aug-PEData Type=Synthetic, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=12026.02 | 13.71 | — | — | — | — | — | — | |
| RPSGModel=DeepSeek-R12026.04 | 13.1 | — | — | — | — | 4.66 | 105 | |
| AUG-PEModel=DeepSeek-R12026.04 | 10.3 | — | — | — | — | 5.17 | 179 | |
| DP-FTData Type=Private, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=42026.02 | 9 | — | — | — | — | — | — | |
| DP-FTData Type=Private, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=22026.02 | 8 | — | — | — | — | — | — | |
| DP-FTData Type=Private, Size=2,000, Backbone=BERTSmall, Privacy Budget (epsilon)=12026.02 | 6 | — | — | — | — | — | — | |
| Aug-PEData Type=Synthetic, Size=2,0002026.02 | — | 13.96 | 14.41 | 14.16 | 13.71 | — | — | |
| DP-FTData Type=Private / full data, Size=75,3162026.02 | — | 44.88 | 18.41 | 17.99 | 17.63 | — | — | |
| DP-FTData Type=Private, Size=2,0002026.02 | — | 20.79 | 9 | 8 | 6 | — | — | |
| DP-RFTData Type=Synthetic, Size=2,0002026.02 | — | 17.46 | 17.81 | 17.32 | 16.79 | — | — | |
| QWENData Type=Synthetic, Size=2,0002026.02 | — | 16.88 | 16.88 | 16.88 | 16.88 | — | — |