Privacy Violation Detection on PKU-SafeRLHF
87.5AccDual-agent
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Dual-agentBackbone=Qwen-Plus2025.12 | 87.5 | 83.2 | 94 | 88.3 | 88 | 0.713 | 0.708 | |
| Dual-agentBackbone=GPT-4o-mini2025.12 | 87 | 82.5 | 94 | 87.9 | 89.8 | 0.736 | 0.75 | |
| Single agentBackbone=Qwen-Plus2025.12 | 82 | 90 | 72 | 80 | 83.5 | 0.644 | 0.668 | |
| Dual-agentBackbone=DeepSeek-R12025.12 | 82 | 90 | 72 | 80 | 87.8 | 0.716 | 0.707 | |
| Single agentBackbone=GPT-4o-mini2025.12 | 81.5 | 81.2 | 82 | 81.6 | 81.9 | 0.63 | 0.63 | |
| Single agentBackbone=DeepSeek-R12025.12 | 79.5 | 86.4 | 70 | 77.3 | 80.5 | 0.601 | 0.585 | |
| MADBackbone=DeepSeek-R12025.12 | 74.5 | 67.4 | 95 | 78.8 | 85.4 | 0.652 | 0.651 | |
| Rule-based2025.12 | 70 | 86.4 | 49.5 | 63 | 70.6 | 0.452 | 0.452 | |
| Zero-shot2025.12 | 69.5 | 64 | 93.2 | 75.9 | 81.5 | 0.545 | 0.535 |