Code Generation on CodeEval-Pro BigCodeBench-Lite-Pro and HumanEval-Pro (1st Pass)
66.7Average AccuracyMEMPROBE
Evaluation Results
| Method | Links | |
|---|---|---|
| MEMPROBEStream Design=Compositional2026.06 | 66.7 | |
| ExpRAGStream Design=Compositional2026.06 | 62.5 | |
| Self-invokingStream Design=Compositional2026.06 | 60.4 | |
| Self-invoking (w/ subtask solution)Stream Design=Compositional2026.06 | 59 | |
| ReMemStream Design=Compositional2026.06 | 58.3 | |
| DC-RSStream Design=Compositional2026.06 | 48.6 | |
| ReActStream Design=Compositional2026.06 | 44.8 | |
| ReActStream Design=Naive2026.06 | 44.8 | |
| AWMStream Design=Naive2026.06 | 44.5 | |
| LangMemStream Design=Compositional2026.06 | 44.4 | |
| Mem0Stream Design=Compositional2026.06 | 43.8 | |
| Mem0Stream Design=Naive2026.06 | 43.8 | |
| MEMPROBEStream Design=Naive2026.06 | 43.1 | |
| ReMemStream Design=Naive2026.06 | 41.7 | |
| AWMStream Design=Compositional2026.06 | 39.6 | |
| LangMemStream Design=Naive2026.06 | 39.6 | |
| ExpRAGStream Design=Naive2026.06 | 39.6 | |
| DC-RSStream Design=Naive2026.06 | 35.4 |