Question Answering on HotpotQA (test) (Acc and R-L Metrics)
90.2AccuracyLATENTGRAPHMEM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LATENTGRAPHMEMBackbone=Qwen3-8B2026.01 | 90.2 | 71.21 | |
| LATENTGRAPHMEMBackbone=SmolLM3-3B2026.01 | 88.9 | 61.39 | |
| PREMemBackbone=SmolLM3-3B2026.01 | 87.7 | 60 | |
| LATENTGRAPHMEMBackbone=Qwen2.5-1.5B-Instruct2026.01 | 86.6 | 58.54 | |
| PREMemBackbone=Qwen3-8B2026.01 | 81.46 | 56.53 | |
| Reasoner-onlyBackbone=Qwen2.5-1.5B-Instruct2026.01 | 80.2 | 52.2 | |
| Reasoner-onlyBackbone=SmolLM3-3B2026.01 | 77.1 | 51.46 | |
| Reasoner-onlyBackbone=Qwen3-8B2026.01 | 72.6 | 62.48 | |
| MemGenBackbone=Qwen3-8B2026.01 | 72.3 | 71.03 | |
| Mem0Backbone=Qwen2.5-1.5B-Instruct2026.01 | 63.8 | 44.08 | |
| MemGenBackbone=SmolLM3-3B2026.01 | 58 | 53.64 | |
| MemGenBackbone=Qwen2.5-1.5B-Instruct2026.01 | 57.1 | 57.59 | |
| A-MemBackbone=SmolLM3-3B2026.01 | 51.3 | 61.18 | |
| PREMemBackbone=Qwen2.5-1.5B-Instruct2026.01 | 49.7 | 48.78 | |
| Mem0Backbone=Qwen3-8B2026.01 | 48.6 | 29.43 | |
| A-MemBackbone=Qwen2.5-1.5B-Instruct2026.01 | 46.6 | 56.73 | |
| Mem0Backbone=SmolLM3-3B2026.01 | 44.9 | 29.3 | |
| THEANINEBackbone=SmolLM3-3B2026.01 | 42.79 | 19.93 | |
| THEANINEBackbone=Qwen3-8B2026.01 | 39.94 | 16.02 | |
| A-MemBackbone=Qwen3-8B2026.01 | 31.4 | 55.1 | |
| RAGBackbone=Qwen3-8B2026.01 | 29.3 | 63.88 | |
| THEANINEBackbone=Qwen2.5-1.5B-Instruct2026.01 | 19.4 | 8.19 | |
| RAGBackbone=SmolLM3-3B2026.01 | 15.3 | 52.22 | |
| RAGBackbone=Qwen2.5-1.5B-Instruct2026.01 | 14.6 | 29.05 |