Single-Hop Question Answering on LoCoMo Single-Hop (test)
37.9F1Reproduced Baselines
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Reproduced BaselinesModel=GPT-oss:20b, Memory=A-mem2026.02 | 37.9 | 32.88 | 0 | |
| PoisonedRAGModel=GPT-oss:20b, Memory=A-mem2026.02 | 35.14 | 30.36 | -7.2 | |
| Reproduced BaselinesModel=Gemma3:27b, Memory=Mem02026.02 | 30.84 | 27.42 | 0 | |
| Reproduced BaselinesModel=Gemma3:27b, Memory=A-mem2026.02 | 30.8 | 26.28 | 0 | |
| Reproduced BaselinesModel=Llama3.2:3b, Memory=A-mem2026.02 | 30.3 | 25 | 0 | |
| PoisonedRAGModel=Gemma3:27b, Memory=Mem02026.02 | 29.03 | 25.74 | -6.1 | |
| PoisonedRAGModel=Gemma3:27b, Memory=A-mem2026.02 | 28.83 | 24.43 | -13.4 | |
| PoisonedRAGModel=Llama3.2:3b, Memory=A-mem2026.02 | 26.27 | 21.64 | -7.1 | |
| PoisonedRAGModel=GPT-oss:20b, Memory=Mem02026.02 | 26.18 | 22.7 | -13.1 | |
| Reproduced BaselinesModel=GPT-oss:20b, Memory=Mem02026.02 | 25.93 | 22.56 | 0 | |
| Reproduced BaselinesModel=Llama3.2:3b, Memory=Mem02026.02 | 25.46 | 22.01 | 0 | |
| PoisonedRAGModel=Llama3.2:3b, Memory=Mem02026.02 | 24.49 | 21.15 | -4.2 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=1 Memory / Question, Memory=A-mem2026.02 | 19.32 | 16.88 | -48.4 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=2 Memories / Question, Memory=A-mem2026.02 | 16.66 | 14.25 | -55.5 | |
| ER-MIAModel=Gemma3:27b, Attack Size=1 Memory / Question, Memory=A-mem2026.02 | 15 | 12.48 | -46.5 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=1 Memory / Question, Memory=A-mem2026.02 | 14.99 | 12.28 | -40.2 | |
| ER-MIAModel=Gemma3:27b, Attack Size=1 Memory / Question, Memory=Mem02026.02 | 13.46 | 11.96 | -48.4 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=1 Memory / Question, Memory=Mem02026.02 | 12.5 | 10.7 | -44.1 | |
| ER-MIAModel=Gemma3:27b, Attack Size=2 Memories / Question, Memory=Mem02026.02 | 11.8 | 10.51 | -53.6 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=1 Memory / Question, Memory=Mem02026.02 | 11.11 | 9.63 | -51.6 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=2 Memories / Question, Memory=A-mem2026.02 | 10.64 | 8.38 | -46.8 | |
| ER-MIAModel=Gemma3:27b, Attack Size=2 Memories / Question, Memory=A-mem2026.02 | 10.4 | 8.53 | -61.6 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=2 Memories / Question, Memory=Mem02026.02 | 9.68 | 8.32 | -56.7 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=2 Memories / Question, Memory=Mem02026.02 | 8.87 | 7.81 | -53 |