Privacy Extraction on PersonaMem v2 (test)
0.9448F1 ScoreMemPrivacy-4B-RL
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| MemPrivacy-4B-RLModel Category=Privacy Models, Model Scale=4B, Training Stage=RL2026.05 | 0.9448 | 0.9454 | 0.9511 | 0.44 | |
| MemPrivacy-4B-SFTModel Category=Privacy Models, Model Scale=4B, Training Stage=SFT2026.05 | 0.9443 | 0.9452 | 0.95 | 0.67 | |
| MemPrivacy-1.7B-RLModel Category=Privacy Models, Model Scale=1.7B, Training Stage=RL2026.05 | 0.9402 | 0.9439 | 0.942 | 0.48 | |
| MemPrivacy-0.6B-RLModel Category=Privacy Models, Model Scale=0.6B, Training Stage=RL2026.05 | 0.934 | 0.9407 | 0.9331 | 0.43 | |
| MemPrivacy-1.7B-SFTModel Category=Privacy Models, Model Scale=1.7B, Training Stage=SFT2026.05 | 0.9316 | 0.9365 | 0.9321 | 0.59 | |
| DeepSeek-V3.2-ThinkModel Category=General Models2026.05 | 0.9218 | 0.9216 | 0.9284 | 39.6 | |
| MemPrivacy-0.6B-SFTModel Category=Privacy Models, Model Scale=0.6B, Training Stage=SFT2026.05 | 0.9208 | 0.9278 | 0.9193 | 0.7 | |
| GPT-5.2Model Category=General Models2026.05 | 0.8806 | 0.877 | 0.8948 | 3.23 | |
| GLM-5Model Category=General Models2026.05 | 0.8787 | 0.8797 | 0.8827 | 8.56 | |
| Kimi-K2Model Category=General Models2026.05 | 0.8735 | 0.8839 | 0.8694 | 1.81 | |
| LongCat-Flash-ChatModel Category=General Models2026.05 | 0.8679 | 0.8716 | 0.8691 | 1.75 | |
| DeepSeek-V3.2Model Category=General Models2026.05 | 0.8661 | 0.8693 | 0.8656 | 3.55 | |
| Gemini-3.1-ProModel Category=General Models2026.05 | 0.8659 | 0.8668 | 0.8682 | 29.21 | |
| Qwen3-4BModel Category=General Models, Model Scale=4B2026.05 | 0.8636 | 0.8638 | 0.8699 | 0.64 | |
| DeepSeek-V3.1-TerminusModel Category=General Models2026.05 | 0.8591 | 0.8623 | 0.8584 | 1.68 | |
| OpenAI-Privacy-FilterModel Category=Privacy Models, Type=Token-classification2026.05 | 0.8527 | 0.858 | 0.8575 | 0.24 | |
| Qwen3-0.6BModel Category=General Models, Model Scale=0.6B2026.05 | 0.0932 | 0.0805 | 0.1349 | 2.36 | |
| Qwen3-1.7BModel Category=General Models, Model Scale=1.7B2026.05 | 0.0692 | 0.0597 | 0.1053 | 2.17 |