Code Generation on BigCodeBench Instruct Full (train)
83.3Last SRAPEX-EM + Opus Judge (A5, E10)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| APEX-EM + Opus Judge (A5, E10)Backbone=Sonnet 4.5, Epochs=10, Variant=A5, Judge=Opus2026.03 | 83.3 | 84 | |
| APEX-EM (EG2, E10)Backbone=Sonnet 4.5, Epochs=10, Variant=EG22026.03 | 81.1 | 82.7 | |
| APEX-EM (E10)Backbone=GPT-4o, Epochs=102026.03 | 81.1 | 81.5 | |
| MemRLBackbone=GPT-4o2026.03 | 59.5 | 62.7 | |
| MemPBackbone=GPT-4o2026.03 | 57.8 | 60.2 | |
| No Memory (A0)Backbone=Sonnet 4.5, Epochs=12026.03 | 53.9 | — | |
| Mem0Backbone=GPT-4o2026.03 | 53 | 57.7 | |
| Self-RAGBackbone=GPT-4o2026.03 | 50 | 55.8 | |
| No MemoryBackbone=GPT-4o2026.03 | 48.5 | — | |
| RAGBackbone=GPT-4o2026.03 | 47.9 | 53 | |
| Pass@10Backbone=GPT-4o2026.03 | — | 57.7 | |
| ReflexionBackbone=GPT-4o2026.03 | — | 58.2 |