On-device LLM Agent Task Drift Evaluation (15 Seeds, ASR Injection)
62.1Task Accuracyfull-mem oracle
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| full-mem oracleBackbone=Qwen2.5-3B, Model state=frozen, Number of seeds=15 (5 for ASR), Injection setting (np)=42026.06 | 62.1 | 46.8 | 287 | 1,068 | 100 | |
| CURATOR (ρ)Backbone=Qwen2.5-3B, Model state=frozen, Number of seeds=15 (5 for ASR), Injection setting (np)=42026.06 | 60.5 | 42 | 107 | 454 | 0 | |
| MaRS-portBackbone=Qwen2.5-3B, Model state=frozen, Number of seeds=15 (5 for ASR), Injection setting (np)=42026.06 | 58.6 | 37.3 | 107 | 1,068 | 75 | |
| recencyBackbone=Qwen2.5-3B, Model state=frozen, Number of seeds=15 (5 for ASR), Injection setting (np)=42026.06 | 55.6 | 32.9 | 107 | 1,068 | 75 | |
| naive-LRUBackbone=Qwen2.5-3B, Model state=frozen, Number of seeds=15 (5 for ASR), Injection setting (np)=42026.06 | 55.4 | 33.7 | 107 | 1,068 | 75 | |
| no-curationBackbone=Qwen2.5-3B, Model state=frozen, Number of seeds=15 (5 for ASR), Injection setting (np)=42026.06 | 52.8 | 29.4 | 287 | 1,068 | 75 | |
| Agentic MemoryBackbone=Qwen2.5-3B, Model state=frozen, Number of seeds=15 (5 for ASR), Injection setting (np)=42026.06 | 49.1 | 26.2 | 287 | 1,068 | 85 | |
| A-MEMBackbone=Qwen2.5-3B, Model state=frozen, Number of seeds=15 (5 for ASR), Injection setting (np)=42026.06 | 48.3 | 24.7 | 287 | 1,068 | 85 |