Long-horizon conversation utility evaluation on PerltQA
80.62AccuracyA-Mem
Evaluation Results
| Method | Links | |
|---|---|---|
| A-MemBase LLM=GPT-4.1-mini, Judge LLM=GPT-4.1, # Avg. Token=72442026.05 | 80.62 | |
| MEMGUARD (2 hop)Base LLM=GPT-4.1-mini, Judge LLM=GPT-4.1, # Avg. Token=15402026.05 | 80.07 | |
| MEMGUARDBase LLM=GPT-4.1-mini, Judge LLM=GPT-4.1, # Avg. Token=16052026.05 | 79.44 | |
| A-MemBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=72442026.05 | 77.04 | |
| MEMGUARD (2 hop)Base LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=12502026.05 | 75.64 | |
| MEMGUARDBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=12972026.05 | 75.31 |