Privacy Extraction on MemPrivacy-Bench v1 (test)
85.97F1 ScoreMemPrivacy-4B-RL
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MemPrivacy-4B-RLModel Category=Privacy Models, Model Scale=4B, Training Stage=RL2026.05 | 85.97 | 86.86 | 87.15 | 2.05 | |
| MemPrivacy-4B-SFTModel Category=Privacy Models, Model Scale=4B, Training Stage=SFT2026.05 | 85.64 | 87.45 | 86.04 | 2.64 | |
| MemPrivacy-0.6B-RLModel Category=Privacy Models, Model Scale=0.6B, Training Stage=RL2026.05 | 84.66 | 86.35 | 85.29 | 1.63 | |
| MemPrivacy-1.7B-RLModel Category=Privacy Models, Model Scale=1.7B, Training Stage=RL2026.05 | 84.54 | 85.21 | 86.06 | 1.81 | |
| MemPrivacy-1.7B-SFTModel Category=Privacy Models, Model Scale=1.7B, Training Stage=SFT2026.05 | 84.22 | 86.14 | 84.44 | 2.14 | |
| MemPrivacy-0.6B-SFTModel Category=Privacy Models, Model Scale=0.6B, Training Stage=SFT2026.05 | 83.09 | 85.67 | 82.84 | 1.96 | |
| Gemini-3.1-ProModel Category=General Models2026.05 | 78.41 | 78.66 | 81.22 | 32.87 | |
| DeepSeek-V3.2-ThinkModel Category=General Models2026.05 | 75.04 | 76.46 | 76.13 | 96.14 | |
| GLM-5Model Category=General Models2026.05 | 70.1 | 70.13 | 73.29 | 11.3 | |
| Kimi-K2Model Category=General Models2026.05 | 69.35 | 73.58 | 69.08 | 5.18 | |
| GPT-5.2Model Category=General Models2026.05 | 68.99 | 65.4 | 78.13 | 4.62 | |
| DeepSeek-V3.1-TerminusModel Category=General Models2026.05 | 66.01 | 69.57 | 65.86 | 5.34 | |
| DeepSeek-V3.2Model Category=General Models2026.05 | 65.91 | 68.07 | 67.19 | 7.03 | |
| LongCat-Flash-ChatModel Category=General Models2026.05 | 64.82 | 65.56 | 67.46 | 4.32 | |
| Qwen3-4BModel Category=General Models, Model Scale=4B2026.05 | 59.34 | 57.29 | 66.78 | 2.72 | |
| OpenAI-Privacy-FilterModel Category=Privacy Models, Type=Token-classification2026.05 | 35.5 | 39.96 | 34.3 | 0.34 | |
| Qwen3-1.7BModel Category=General Models, Model Scale=1.7B2026.05 | 30.83 | 26.48 | 43.85 | 3.16 | |
| Qwen3-0.6BModel Category=General Models, Model Scale=0.6B2026.05 | 26.43 | 21.73 | 41.58 | 3.05 |