Open-Domain Question Answering on LoCoMo Open-Domain (test)
15.12F1 ScoreReproduced Baselines
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Reproduced BaselinesModel=GPT-oss:20b, Memory=A-mem2026.02 | 15.12 | 15.87 | 0 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=2 Memories / Question, Memory=A-mem2026.02 | 14.06 | 13.44 | -46.8 | |
| Reproduced BaselinesModel=Gemma3:27b, Memory=Mem02026.02 | 12.22 | 10.3 | 0 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=1 Memory / Question, Memory=A-mem2026.02 | 12.14 | 10.93 | -40.2 | |
| PoisonedRAGModel=GPT-oss:20b, Memory=A-mem2026.02 | 12.06 | 9.78 | -7.2 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=1 Memory / Question, Memory=A-mem2026.02 | 11.45 | 11.11 | -48.4 | |
| Reproduced BaselinesModel=GPT-oss:20b, Memory=Mem02026.02 | 11.09 | 8.56 | 0 | |
| Reproduced BaselinesModel=Llama3.2:3b, Memory=Mem02026.02 | 11.02 | 8.87 | 0 | |
| PoisonedRAGModel=Gemma3:27b, Memory=Mem02026.02 | 10.44 | 8.55 | -6.1 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=1 Memory / Question, Memory=Mem02026.02 | 10.37 | 8.48 | -44.1 | |
| Reproduced BaselinesModel=Llama3.2:3b, Memory=A-mem2026.02 | 8.96 | 8.2 | 0 | |
| ER-MIAModel=Gemma3:27b, Attack Size=1 Memory / Question, Memory=A-mem2026.02 | 8.81 | 7.03 | -46.5 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=2 Memories / Question, Memory=A-mem2026.02 | 8.79 | 8.64 | -55.5 | |
| PoisonedRAGModel=Llama3.2:3b, Memory=A-mem2026.02 | 8.66 | 8.17 | -7.1 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=1 Memory / Question, Memory=Mem02026.02 | 8.62 | 7.3 | -51.6 | |
| PoisonedRAGModel=Llama3.2:3b, Memory=Mem02026.02 | 8.6 | 7.51 | -4.2 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=2 Memories / Question, Memory=Mem02026.02 | 8.59 | 7.73 | -53 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=2 Memories / Question, Memory=Mem02026.02 | 8.52 | 7.61 | -56.7 | |
| ER-MIAModel=Gemma3:27b, Attack Size=2 Memories / Question, Memory=Mem02026.02 | 8.24 | 7.2 | -53.6 | |
| ER-MIAModel=Gemma3:27b, Attack Size=1 Memory / Question, Memory=Mem02026.02 | 8.07 | 6.64 | -48.4 | |
| Reproduced BaselinesModel=Gemma3:27b, Memory=A-mem2026.02 | 7.97 | 6.45 | 0 | |
| PoisonedRAGModel=GPT-oss:20b, Memory=Mem02026.02 | 7.08 | 6.95 | -13.1 | |
| PoisonedRAGModel=Gemma3:27b, Memory=A-mem2026.02 | 6.12 | 4.48 | -13.4 | |
| ER-MIAModel=Gemma3:27b, Attack Size=2 Memories / Question, Memory=A-mem2026.02 | 6.07 | — | -61.6 |