Memory-based Question Answering on LoCoMo-Plus (n=401)
93.3Accuracy (%)Kumiho (4o answer)
Evaluation results
| Method | Links | |
|---|---|---|
| Kumiho (4o answer)Model=GPT-4o2026.03 | 93.3 | |
| Kumiho (4o-mini answer)Model=GPT-4o-mini2026.03 | 88 | |
| Gemini 2.5 Pro (1M)Model=Gemini 2.5 Pro2026.03 | 45.7 | |
| Gemini 2.5 Flash (1M)Model=Gemini 2.5 Flash2026.03 | 44.6 | |
| GPT-4.1 (full context)Model=GPT-4.12026.03 | 43.6 | |
| SeComModel=Various2026.03 | 42.6 | |
| A-MEMModel=Various2026.03 | 42.4 | |
| GPT-4o (full context)Model=GPT-4o2026.03 | 41.9 | |
| Mem0Model=Various2026.03 | 41.4 | |
| RAG (text-embed-large)Model=text-embed-large2026.03 | 29.8 | |
| RAG (text-embed-small)Model=text-embed-small2026.03 | 24.9 | |
| RAG (text-ada-002)Model=text-ada-0022026.03 | 23.5 |