Code Generation on CodeEval-Pro BigCodeBench-Lite-Pro and HumanEval-Pro (2nd Pass)
64.6Average AccuracyMEMPROBE
Evaluation results
| Method | Links | |
|---|---|---|
| MEMPROBEStream Design=Compositional2026.06 | 64.6 | |
| ExpRAGStream Design=Compositional2026.06 | 62.5 | |
| Self-invokingStream Design=Compositional2026.06 | 60.4 | |
| Self-invoking (w/ subtask solution)Stream Design=Compositional2026.06 | 59 | |
| ReMemStream Design=Compositional2026.06 | 56.3 | |
| DC-RSStream Design=Compositional2026.06 | 50.7 | |
| LangMemStream Design=Compositional2026.06 | 45.8 | |
| ReActStream Design=Compositional2026.06 | 44.8 | |
| ReActStream Design=Naive2026.06 | 44.8 | |
| MEMPROBEStream Design=Naive2026.06 | 44.5 | |
| Mem0Stream Design=Compositional2026.06 | 44.4 | |
| ReMemStream Design=Naive2026.06 | 43.8 | |
| LangMemStream Design=Naive2026.06 | 42.4 | |
| Mem0Stream Design=Naive2026.06 | 42.4 | |
| AWMStream Design=Naive2026.06 | 41 | |
| DC-RSStream Design=Naive2026.06 | 39.6 | |
| ExpRAGStream Design=Naive2026.06 | 39.6 | |
| AWMStream Design=Compositional2026.06 | 38.2 |