Privacy and Helpfulness evaluation on PrivacyLens judged by GPT-5.5 w/ high reasoning (held out)
38.3Leakage RateGPT-5.4-mini
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-5.4-miniReasoning mode=high2026.06 | 38.3 | 2.79 | |
| Nemotron-3-Nano-4B + RL (PrivacyAlign)Reasoning mode=thinking, RL training=annotation-conditioned reward2026.06 | 38.3 | 2.06 | |
| GPT-5.5Reasoning mode=high2026.06 | 39.8 | 2.88 | |
| Gemini-3.1-Flash-LiteReasoning mode=high2026.06 | 46 | 2.46 | |
| Nemotron-3-Nano-4BReasoning mode=thinking2026.06 | 49.3 | 1.91 | |
| Gemini-3.1-ProReasoning mode=high2026.06 | 51.1 | 2.74 | |
| Qwen3-4B + RL (PrivacyAlign)Reasoning mode=thinking, RL training=annotation-conditioned reward2026.06 | 51.9 | 2.25 | |
| Qwen3-4BReasoning mode=thinking2026.06 | 54 | 1.78 | |
| Qwen3-8B + RL (PrivacyAlign)Reasoning mode=thinking, RL training=annotation-conditioned reward2026.06 | 55 | 2.34 | |
| Qwen3-8BReasoning mode=thinking2026.06 | 57.4 | 1.9 |