Safety Evaluation on PS-Bench
29.1ASR (Hate Speech)MemU (with intent-legitimation suppression)
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| MemU (with intent-legitimation suppression)agent_framework=MemU, intervention=intent-legitimation suppression, base_model=GPT-4o-mini2026.01 | 29.1 | 12.9 | 11 | 11.2 | 37.5 | 16.5 | 30.8 | 6.2 | 19.4 | |
| MemOS (with intent-legitimation suppression)agent_framework=MemOS, intervention=intent-legitimation suppression, base_model=GPT-4o-mini2026.01 | 29.7 | 14.4 | 10.8 | 10.5 | 37.8 | 18.6 | 30.2 | 5 | 19.63 | |
| Amem (with intent-legitimation suppression)agent_framework=Amem, intervention=intent-legitimation suppression, base_model=GPT-4o-mini2026.01 | 32.1 | 14.2 | 13.1 | 15.5 | 43.3 | 24.3 | 42.4 | 8.2 | 24.14 | |
| Statelessagent_framework=Stateless, intervention=none, base_model=GPT-4o-mini2026.01 | 36 | 16 | 11 | 6 | 52 | 22 | 34 | 8 | 23.13 | |
| MemOSagent_framework=MemOS, intervention=none, base_model=GPT-4o-mini2026.01 | 40.3 | 21.8 | 18.4 | 17.8 | 47.2 | 31 | 45.7 | 7.6 | 28.73 | |
| MemUagent_framework=MemU, intervention=none, base_model=GPT-4o-mini2026.01 | 42.2 | 16.6 | 19.6 | 18 | 51.6 | 29.2 | 43.3 | 7.2 | 28.46 | |
| Amemagent_framework=Amem, intervention=none, base_model=GPT-4o-mini2026.01 | 44.9 | 19.4 | 19.1 | 19.3 | 50.3 | 30.4 | 44.5 | 9.4 | 29.66 |