Lifelong Agent Bench OS Task
78.8Success Rate (Last Epoch)MemRL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MemRLModel=GPT-4o-mini2026.01 | 78.8 | 80.4 | |
| SkillMASEvaluation Source=Internal evaluation2026.05 | 76.7 | — | |
| MemPModel=GPT-4o-mini2026.01 | 73.6 | 74.2 | |
| Traj-BootstrapEvaluation Source=Internal evaluation/rerun2026.05 | 70 | — | |
| RAGModel=GPT-4o-mini2026.01 | 69 | 70 | |
| CDMemEvaluation Source=Internal evaluation/rerun2026.05 | 68 | — | |
| No MemoryModel=GPT-4o-mini2026.01 | 67.4 | — | |
| Mem0Model=GPT-4o-mini2026.01 | 67 | 70.2 | |
| OPD-Evolver-9BBackbone LLM=QWEN3.5-9B2026.06 | 65 | — | |
| Self-RAGModel=GPT-4o-mini2026.01 | 64.6 | 73.2 | |
| QWEN3.5-397B-A17BBackbone LLM=QWEN3.5-397B-A17B2026.06 | 63 | — | |
| ReActEvaluation Source=Internal evaluation/rerun2026.05 | 62 | — | |
| MemEvolveBackbone LLM=QWEN3.5-9B2026.06 | 61 | — | |
| EvolveRBackbone LLM=QWEN3.5-9B2026.06 | 59.5 | — | |
| Direct LLMEvaluation Source=Internal evaluation/rerun2026.05 | 59.3 | — | |
| STEP-3.5-FLASH (196B)Backbone LLM=STEP-3.5-FLASH (196B)2026.06 | 58 | — | |
| MempBackbone LLM=QWEN3.5-9B2026.06 | 56 | — | |
| ReasoningBankBackbone LLM=QWEN3.5-9B2026.06 | 55 | — | |
| No MemoryBackbone LLM=QWEN3.5-9B2026.06 | 52.5 | — | |
| CheatsheetBackbone LLM=QWEN3.5-9B2026.06 | 52.5 | — | |
| OPD-Evolver-4BBackbone LLM=QWEN3-4B2026.06 | 49.5 | — | |
| EvolveRBackbone LLM=QWEN3-4B2026.06 | 46.5 | — | |
| ExpeLBackbone LLM=QWEN3.5-9B2026.06 | 46.5 | — | |
| MemEvolveBackbone LLM=QWEN3-4B2026.06 | 44 | — | |
| AWMBackbone LLM=QWEN3.5-9B2026.06 | 44 | — | |
| MempBackbone LLM=QWEN3-4B2026.06 | 41 | — | |
| CheatsheetBackbone LLM=QWEN3-4B2026.06 | 38.5 | — | |
| ReasoningBankBackbone LLM=QWEN3-4B2026.06 | 38 | — | |
| No MemoryBackbone LLM=QWEN3-4B2026.06 | 36.5 | — | |
| AWMBackbone LLM=QWEN3-4B2026.06 | 36 | — | |
| ExpeLBackbone LLM=QWEN3-4B2026.06 | 34.5 | — | |
| Pass@10Model=GPT-4o-mini2026.01 | — | 75.6 |