Targeted training data extraction on LM-Extraction (test)
62.8RecallETHICIST
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ETHICISTBackbone=GPT-Neo 1.3B, prompt tuning=true, smooth loss=true, calibrated confidence=true2023.07 | 62.8 | 53.8 | |
| ETHICISTBackbone=GPT-Neo 1.3B, prompt tuning=true, smooth loss=false, calibrated confidence=false, confidence estimation=Comparing (LM)2023.07 | 62.4 | 48.7 | |
| ETHICISTBackbone=GPT-Neo 1.3B, prompt tuning=true, smooth loss=true, calibrated confidence=false2023.07 | 62.3 | 47.5 | |
| ETHICISTBackbone=GPT-Neo 1.3B, prompt tuning=true, smooth loss=false, calibrated confidence=true2023.07 | 61.2 | 52.4 | |
| ETHICISTBackbone=GPT-Neo 1.3B, prompt tuning=true, smooth loss=false, calibrated confidence=false, confidence estimation=perplexity2023.07 | 60.8 | 44.6 | |
| Comparing (LM)Backbone=GPT-Neo 1.3B2023.07 | 51.9 | 37.4 | |
| Comparing (lowercase)Backbone=GPT-Neo 1.3B2023.07 | 51.5 | 32.5 | |
| PerplexityBackbone=GPT-Neo 1.3B2023.07 | 51.3 | 32.2 | |
| ETHICISTBackbone=GPT-Neo 1.3B, prompt tuning=false, calibrated confidence=true2023.07 | 50.9 | 38 | |
| Comparing (zlib)Backbone=GPT-Neo 1.3B2023.07 | 49.7 | 25.6 |