Code Generation on CodeEval-Pro BigCodeBench-Lite-Pro and HumanEval-Pro (Held-out)
79.2Average AccuracySelf-invoking
Evaluation results
| Method | Links | |
|---|---|---|
| Self-invokingStream Design=Naive2026.06 | 79.2 | |
| Self-invoking (w/ subtask solution)Stream Design=Naive2026.06 | 73.3 | |
| ReActStream Design=Compositional2026.06 | 72.5 | |
| ReActStream Design=Naive2026.06 | 72.5 | |
| MEMPROBEStream Design=Compositional2026.06 | 71.7 | |
| LangMemStream Design=Compositional2026.06 | 70.8 | |
| Mem0Stream Design=Compositional2026.06 | 70.8 | |
| MEMPROBEStream Design=Naive2026.06 | 70.8 | |
| AWMStream Design=Compositional2026.06 | 70 | |
| AWMStream Design=Naive2026.06 | 70 | |
| ReMemStream Design=Naive2026.06 | 70 | |
| Mem0Stream Design=Naive2026.06 | 69.2 | |
| ExpRAGStream Design=Naive2026.06 | 69.2 | |
| ExpRAGStream Design=Compositional2026.06 | 68.3 | |
| ReMemStream Design=Compositional2026.06 | 67.5 | |
| LangMemStream Design=Naive2026.06 | 66.7 | |
| DC-RSStream Design=Naive2026.06 | 65.8 | |
| DC-RSStream Design=Compositional2026.06 | 60.8 |