Agent Memory Question Answering on PerLTQA (test)
42.68BLEUxMemory
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| xMemoryModel=Llama-3.1-8B-Ins2026.02 | 42.68 | 52.37 | 47.84 | 6,066.4 | |
| NemoriModel=Llama-3.1-8B-Ins2026.02 | 41.01 | 49.62 | 44.65 | 11,440.04 | |
| xMemoryModel=GPT-5 nano2026.02 | 36.79 | 46.23 | 41.25 | 7,307.3 | |
| xMemoryModel=Qwen3-8B2026.02 | 36.24 | 47.08 | 42.5 | 5,087.18 | |
| MemoryOSModel=Qwen3-8B2026.02 | 35.14 | 42.35 | 38.48 | 6,499.47 | |
| MemoryOSModel=Llama-3.1-8B-Ins2026.02 | 35.14 | 42.35 | 38.48 | 6,511.11 | |
| Naive RAGModel=Llama-3.1-8B-Ins2026.02 | 33.67 | 44.84 | 39.65 | 9,531.3 | |
| NemoriModel=GPT-5 nano2026.02 | 33.44 | 41.79 | 38.43 | 11,882.7 | |
| A-MemModel=GPT-5 nano2026.02 | 33.12 | 41.17 | 37.94 | 14,717.77 | |
| NemoriModel=Qwen3-8B2026.02 | 32.55 | 42.8 | 38.05 | 9,091.72 | |
| Naive RAGModel=Qwen3-8B2026.02 | 32.08 | 41.37 | 35.95 | 6,274.38 | |
| A-MemModel=Qwen3-8B2026.02 | 28.62 | 34.99 | 33.54 | 9,864.32 | |
| MemoryOSModel=GPT-5 nano2026.02 | 27.66 | 33.83 | 31.44 | 12,669.39 | |
| Naive RAGModel=GPT-5 nano2026.02 | 27.35 | 37.76 | 32.35 | 10,756.39 | |
| LightMemModel=Llama-3.1-8B-Ins2026.02 | 23.47 | 35.93 | 33.65 | 5,452.49 | |
| LightMemModel=Qwen3-8B2026.02 | 16.97 | 32.41 | 29.87 | 7,692.05 | |
| LightMemModel=GPT-5 nano2026.02 | 14.04 | 28.3 | 26.11 | 7,579.27 | |
| A-MemModel=Llama-3.1-8B-Ins2026.02 | 11.4 | 13.28 | 15.9 | 7,707.4 |