Code Generation on MBPP (MBPP metric)
70.2MBPP ScoreOPTIMER
Evaluation Results
| Method | Links | |
|---|---|---|
| OPTIMERBase Model=Gemma 3 27B2026.03 | 70.2 | |
| OPTIMERBase Model=SEA-LION v42026.03 | 66 | |
| SEA-LION v4 27B IT2026.03 | 57.6 | |
| Gemma 3 27B IT2026.03 | 37.8 |
Loading the SOTA2 catalog…