Question Answering on PersonaMem v1
52.74R-Fact ScoreRAG (top-8)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| RAG (top-8)Backbone=Qwen3-8B, Extra (Parameters)=0, Ctx (Context Length)=2232.0, KV (MB)=313.88, Peak (MB)=16177.5, Lat. (s)=1.792026.06 | 52.74 | 20.43 | 54.03 | 41.82 | 64.85 | 46.88 | 44.5 | |
| Full-contextBackbone=Qwen3-8B, Extra (Parameters)=0, Ctx (Context Length)=25943.2, KV (MB)=3648.27, Peak (MB)=23001.9, Lat. (s)=3.812026.06 | 46.6 | 6.5 | 73.4 | 41.8 | 81.8 | 59.6 | 53.3 | |
| LightMem (top-50)Backbone=Qwen3-8B, Extra (Parameters)=0, Ctx (Context Length)=2523.8, KV (MB)=354.77, Peak (MB)=16248.1, Lat. (s)=1.672026.06 | 44.52 | 23.65 | 35.25 | 54.54 | 54.54 | 45.61 | 41.77 | |
| LPM (Ours)Backbone=Qwen3-8B, Extra (Parameters)=1.3 M, Ctx (Context Length)=411.5, KV (MB)=56.73, Peak (MB)=16216.8, Lat. (s)=0.882026.06 | 35.62 | 3.23 | 66.91 | 45.45 | 80.81 | 54.39 | 48.22 | |
| LoRABackbone=Qwen3-8B, Extra (Parameters)=154 M, Ctx (Context Length)=403.5, KV (MB)=56.73, Peak (MB)=15671.4, Lat. (s)=5.442026.06 | 33.6 | 17.2 | 61.2 | 30.9 | 74.7 | 35.1 | 44.3 | |
| Prompt TuningBackbone=Qwen3-8B, Extra (Parameters)=40 M, Ctx (Context Length)=953.5, KV (MB)=120.0, Peak (MB)=16302.5, Lat. (s)=1.652026.06 | 33.6 | 6.5 | 30.9 | 32.7 | 53.5 | 26.3 | 31.24 | |
| Embedding-to-PrefixBackbone=Qwen3-8B, Extra (Parameters)=346 M, Ctx (Context Length)=411.5, KV (MB)=55.54, Peak (MB)=16942.1, Lat. (s)=0.362026.06 | 28.08 | 9.68 | 59.71 | 34.55 | 70.71 | 31.58 | 40.75 |