Temporal Question Answering on LoCoMo Temporal (test)
44.09F1 ScoreReproduced Baselines
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Reproduced BaselinesModel=GPT-oss:20b, Memory=A-mem2026.02 | 44.09 | 32.59 | 0 | |
| PoisonedRAGModel=GPT-oss:20b, Memory=A-mem2026.02 | 41.66 | 31.33 | -7.2 | |
| Reproduced BaselinesModel=GPT-oss:20b, Memory=Mem02026.02 | 28.59 | 20.85 | 0 | |
| PoisonedRAGModel=Llama3.2:3b, Memory=Mem02026.02 | 27.17 | 20.13 | -4.2 | |
| PoisonedRAGModel=Llama3.2:3b, Memory=A-mem2026.02 | 26.63 | 18.64 | -7.1 | |
| Reproduced BaselinesModel=Gemma3:27b, Memory=A-mem2026.02 | 26.4 | 20.95 | 0 | |
| Reproduced BaselinesModel=Llama3.2:3b, Memory=Mem02026.02 | 26.06 | 20.32 | 0 | |
| Reproduced BaselinesModel=Llama3.2:3b, Memory=A-mem2026.02 | 25.4 | 19.4 | 0 | |
| Reproduced BaselinesModel=Gemma3:27b, Memory=Mem02026.02 | 23.99 | 17.5 | 0 | |
| PoisonedRAGModel=Gemma3:27b, Memory=Mem02026.02 | 22.83 | 16.68 | -6.1 | |
| PoisonedRAGModel=Gemma3:27b, Memory=A-mem2026.02 | 21.72 | 17.55 | -13.4 | |
| PoisonedRAGModel=GPT-oss:20b, Memory=Mem02026.02 | 21.61 | 16.47 | -13.1 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=2 Memories / Question, Memory=A-mem2026.02 | 16.09 | 14.14 | -55.5 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=1 Memory / Question, Memory=A-mem2026.02 | 15.81 | 13.53 | -48.4 | |
| ER-MIAModel=Gemma3:27b, Attack Size=1 Memory / Question, Memory=Mem02026.02 | 13.55 | 11.11 | -48.4 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=1 Memory / Question, Memory=A-mem2026.02 | 13.22 | 10.04 | -40.2 | |
| ER-MIAModel=Gemma3:27b, Attack Size=1 Memory / Question, Memory=A-mem2026.02 | 12.35 | 9.9 | -46.5 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=1 Memory / Question, Memory=Mem02026.02 | 12.04 | 11.05 | -44.1 | |
| ER-MIAModel=Gemma3:27b, Attack Size=2 Memories / Question, Memory=Mem02026.02 | 11.96 | 10.11 | -53.6 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=1 Memory / Question, Memory=Mem02026.02 | 11.11 | 9.15 | -51.6 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=2 Memories / Question, Memory=Mem02026.02 | 10.87 | 9.93 | -53 | |
| ER-MIAModel=GPT-oss:20b, Attack Size=2 Memories / Question, Memory=Mem02026.02 | 9.9 | 8.22 | -56.7 | |
| ER-MIAModel=Llama3.2:3b, Attack Size=2 Memories / Question, Memory=A-mem2026.02 | 9.84 | 7.29 | -46.8 | |
| ER-MIAModel=Gemma3:27b, Attack Size=2 Memories / Question, Memory=A-mem2026.02 | 9.81 | 8.78 | -61.6 |