Code Generation on BigCodeBench Lite-Pro Naive Stream
44.8AccuracyReAct
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ReActEvaluation Pass=1st2026.06 | 44.8 | — | — | |
| ReActEvaluation Pass=2nd2026.06 | 44.8 | — | — | |
| AWMEvaluation Pass=1st2026.06 | 44.5 | 0.3 | — | |
| MEMPROBEEvaluation Pass=2nd2026.06 | 44.5 | — | 1.4 | |
| Mem0Evaluation Pass=1st2026.06 | 43.8 | 1 | — | |
| ReMemEvaluation Pass=2nd2026.06 | 43.8 | — | 2.1 | |
| MEMPROBEEvaluation Pass=1st2026.06 | 43.1 | 1.7 | — | |
| LangMemEvaluation Pass=2nd2026.06 | 42.4 | — | 2.8 | |
| Mem0Evaluation Pass=2nd2026.06 | 42.4 | — | 1.4 | |
| ReMemEvaluation Pass=1st2026.06 | 41.7 | 3.1 | — | |
| AWMEvaluation Pass=2nd2026.06 | 41 | — | 3.5 | |
| LangMemEvaluation Pass=1st2026.06 | 39.6 | 5.2 | — | |
| DC-RSEvaluation Pass=2nd2026.06 | 39.6 | — | 4.2 | |
| ExpRAGEvaluation Pass=1st2026.06 | 39.6 | 5.2 | — | |
| ExpRAGEvaluation Pass=2nd2026.06 | 39.6 | — | 0 | |
| DC-RSEvaluation Pass=1st2026.06 | 35.4 | 9.4 | — |