Privacy-Aware Response Generation on PrivacyAlign (test)
14.5Leaks %GPT-5.5
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-5.5Prompting strategy=PE, Reasoning mode=high2026.06 | 14.5 | 17.5 | 70.7 | 7.5 | |
| Nemotron-3-Nano-4B + RL w/ annotation-conditioned rewardPrompting strategy=PE, Reasoning mode=thinking2026.06 | 17.8 | 55.5 | 33.7 | 1.1 | |
| Nemotron-3-Nano-4B + RL w/ trained gen-RMPrompting strategy=PE, Reasoning mode=thinking2026.06 | 18.1 | 69.9 | 23 | 3.1 | |
| GPT-5.4-miniPrompting strategy=PE, Reasoning mode=high2026.06 | 18.6 | 35.6 | 51.4 | 6.5 | |
| Claude Sonnet 4.6Prompting strategy=PE, Reasoning mode=high2026.06 | 21.1 | 19 | 62.1 | 5.2 | |
| Claude Opus 4.7Prompting strategy=PE, Reasoning mode=high2026.06 | 22.8 | 16.5 | 64.4 | 6.5 | |
| GPT-5.5Prompting strategy=Naive, Reasoning mode=high2026.06 | 23.3 | 18.3 | 63.2 | — | |
| Gemini 3.1 Flash LitePrompting strategy=PE, Reasoning mode=high2026.06 | 24.8 | 47.6 | 39.1 | 3.7 | |
| Nemotron-3-Nano-4B + RL w/ trained gen-RMPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 27.8 | 62.4 | 26.1 | — | |
| GPT-5.4-miniPrompting strategy=Naive, Reasoning mode=high2026.06 | 28.1 | 36.8 | 44.9 | — | |
| Nemotron-3-Nano-4B + RL w/ annotation-conditioned rewardPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 28.6 | 51.2 | 32.6 | — | |
| Claude Sonnet 4.6Prompting strategy=Naive, Reasoning mode=high2026.06 | 28.9 | 17.3 | 56.9 | — | |
| Nemotron-3-Nano-4B + CI-RLPrompting strategy=PE, Reasoning mode=thinking2026.06 | 29.3 | 66.3 | 22.8 | 0.9 | |
| Gemini 3.1 ProPrompting strategy=PE, Reasoning mode=high2026.06 | 30.3 | 32.6 | 47.9 | 10.6 | |
| Claude Opus 4.7Prompting strategy=Naive, Reasoning mode=high2026.06 | 34.1 | 10.8 | 57.9 | — | |
| Qwen3-4B + RL w/ annotation-conditioned rewardPrompting strategy=PE, Reasoning mode=thinking2026.06 | 34.6 | 51.9 | 30.1 | 2.8 | |
| Qwen3-4B + RL w/ trained gen-RMPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 35.1 | 60.1 | 25.8 | — | |
| Qwen3-8B + RL w/ annotation-conditioned rewardPrompting strategy=PE, Reasoning mode=thinking2026.06 | 36.8 | 48.6 | 31.8 | 3.7 | |
| Qwen3-4B + CI-RLPrompting strategy=PE, Reasoning mode=thinking2026.06 | 38.1 | 57.1 | 25.7 | 1.6 | |
| Nemotron-3-Nano-4B + CI-RLPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 38.3 | 61.5 | 21.9 | — | |
| Qwen3-4B + RL w/ trained gen-RMPrompting strategy=PE, Reasoning mode=thinking2026.06 | 38.4 | 65.1 | 21.1 | 4.7 | |
| Gemini 3.1 Flash LitePrompting strategy=Naive, Reasoning mode=high2026.06 | 39.2 | 46 | 35.4 | — | |
| Qwen3-8B + RL w/ trained gen-RMPrompting strategy=PE, Reasoning mode=thinking2026.06 | 39.8 | 59.4 | 21.8 | 3.4 | |
| Qwen3-4B + RL w/ annotation-conditioned rewardPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 40.8 | 51.9 | 27.3 | — | |
| Qwen3-8B + CI-RLPrompting strategy=PE, Reasoning mode=thinking2026.06 | 41.1 | 51.6 | 25.7 | 0 | |
| Qwen3-8B + RL w/ annotation-conditioned rewardPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 41.2 | 48.8 | 28.1 | — | |
| Gemini 3.1 ProPrompting strategy=Naive, Reasoning mode=high2026.06 | 41.4 | 36.1 | 37.3 | — | |
| Qwen3-8B + CI-RLPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 42.9 | 54.2 | 25.7 | — | |
| Qwen3-4B + CI-RLPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 45.2 | 54.4 | 24.1 | — | |
| Qwen3-8B + RL w/ trained gen-RMPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 46.8 | 59.3 | 18.4 | — | |
| Nemotron-3-Nano-4BPrompting strategy=PE, Reasoning mode=thinking2026.06 | 52.9 | 53.2 | 21.6 | 2.5 | |
| Nemotron-3-Nano-4BPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 56.1 | 53.4 | 19.1 | — | |
| Qwen3-4BPrompting strategy=PE, Reasoning mode=thinking2026.06 | 62.9 | 50.4 | 17.3 | 1.6 | |
| Qwen3-4BPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 63 | 49.4 | 18.9 | — | |
| Qwen3-8BPrompting strategy=PE, Reasoning mode=thinking2026.06 | 71.2 | 40.6 | 16.6 | 3.3 | |
| Qwen3-8BPrompting strategy=Naive, Reasoning mode=thinking2026.06 | 74.6 | 41.2 | 13.3 | — |