Question Answering on LoCoMo 1.0 (test)
47F1 (Multi-Hop)Full-ctx 235B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Full-ctx 235BCondition=Full-ctx 235B, Mechanism=All 19 sessions as conversation history2026.03 | 47 | 61.9 | 19.1 | 16.1 | 43.9 | |
| Mem0 (GPT-4o)Condition=Published baseline, Backbone=GPT-4o2026.03 | 38.7 | — | 47.7 | 48.9 | 41 | |
| ZepCondition=Published baseline2026.03 | 35.7 | — | 49.6 | 42 | 36.7 | |
| LangMemCondition=Published baseline2026.03 | 35.5 | — | 40.9 | 30.8 | 33.3 | |
| Warm (compound)Condition=Warm (compound), Mechanism=Memory retrieval + confidence routing2026.03 | 20.8 | 46.4 | 25.5 | 10.3 | 30.5 | |
| Warm (memory only)Condition=Warm (memory only), Mechanism=Memory retrieval to 8B, no routing2026.03 | 20.8 | 45.7 | 25.5 | 10.3 | 30.1 | |
| Cold 8BCondition=Cold 8B, Mechanism=Direct 8B, no memory, no routing2026.03 | 8.8 | 17.5 | 24.7 | 14.1 | 15.4 | |
| Cold 235BCondition=Cold 235B, Mechanism=Direct 235B, no memory, no routing2026.03 | 7.6 | 19.8 | 16.1 | 6.4 | 13.7 | |
| Cold compoundCondition=Cold compound, Mechanism=No memory + confidence routing2026.03 | 6.2 | 16.7 | 9.6 | 13.1 | 13 |