Code Generation on LCB v6 (val)
78Accuracy (%)Base
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BaseBackbone=Qwen3-32B, Generations=642026.04 | 78 | 2.88 | 22.9 | |
| ControlBackbone=Qwen3-32B, Generations=642026.04 | 75.3 | 3.05 | 27.7 | |
| MEMENTOBackbone=Qwen3-32B, Generations=642026.04 | 74 | 1.12 | 9.3 | |
| BaseBackbone=Qwen3-8B, Generations=642026.04 | 73.1 | 1.76 | 15.6 | |
| ControlBackbone=Qwen3-8B, Generations=642026.04 | 70 | 1.89 | 19.2 | |
| Mem. + RLBackbone=Qwen3-8B, Generations=642026.04 | 68.8 | 1.12 | 10.3 | |
| MEMENTOBackbone=Qwen3-8B, Generations=642026.04 | 66.5 | 0.6 | 5.6 | |
| ControlBackbone=Phi-4-r (14B), Generations=642026.04 | 65 | 2.64 | 26.8 | |
| BaseBackbone=Olmo 3 (7B), Generations=642026.04 | 64.5 | 3.33 | 40 | |
| BaseBackbone=Phi-4-r (14B), Generations=642026.04 | 64.1 | 2.45 | 29.2 | |
| MEMENTOBackbone=Phi-4-r (14B), Generations=642026.04 | 61.8 | 0.92 | 9.5 | |
| ControlBackbone=Olmo 3 (7B), Generations=642026.04 | 58.8 | 3.22 | 37.6 | |
| MEMENTOBackbone=Olmo 3 (7B), Generations=642026.04 | 56 | 2.21 | 20.6 |