Long-term conversation recall on LOCOMO full 10-conversation benchmark
79.8LLM-as-Judge AccuracyFull Context (upper bound)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Full Context (upper bound)N (Sample Size)=600, Judge Model=Gemini 3 Flash, Answer Generation Model=Gemini 3 Flash2026.06 | 79.8 | 76.4 | 0.65 | |
| UaC (Gemini)N (Sample Size)=600, Judge Model=Gemini 3 Flash, Answer Generation Model=Gemini 3 Flash2026.06 | 78.8 | 75.4 | — | |
| MemMachineN (Sample Size)=600, Judge Model=Gemini 3 Flash, Answer Generation Model=Gemini 3 Flash2026.06 | 72.7 | 69 | 0.003 | |
| Hindsight (lite)N (Sample Size)=600, Judge Model=Gemini 3 Flash, Answer Generation Model=Gemini 3 Flash2026.06 | 69.7 | 65.9 | 1.5 | |
| EverMemOS (lite)N (Sample Size)=600, Judge Model=Gemini 3 Flash, Answer Generation Model=Gemini 3 Flash2026.06 | 55.5 | 51.5 | 10 | |
| A-MEMN (Sample Size)=600, Judge Model=Gemini 3 Flash, Answer Generation Model=Gemini 3 Flash2026.06 | 51.8 | 47.8 | 10 | |
| Mem0N (Sample Size)=600, Judge Model=Gemini 3 Flash, Answer Generation Model=Gemini 3 Flash2026.06 | 29.3 | 25.8 | 10 |