Code on MBPP Sanitized (pass@1)
85.97pass@1Nemotron-3-Ultra 550B-A55B-Base
Evaluation results
| Method | Links | |
|---|---|---|
| Nemotron-3-Ultra 550B-A55B-Baseshots=3-shot, metric_mode=pass@1, n=32, sanitization=EvalPlus sanitized2026.06 | 85.97 | |
| Mistral-Large-3 675B-Base-2512shots=3-shot, metric_mode=pass@1, n=32, sanitization=EvalPlus sanitized2026.06 | 84.08 | |
| N-3-Super 120B-A12B-BaseShots=3, Sample size (n)=322026.04 | 78.38 | |
| GLM-4.5 Air-BaseShots=3, Sample size (n)=322026.04 | 77.5 | |
| Ling-flash base-2.0Shots=3, Sample size (n)=322026.04 | 77.3 | |
| GLM-4.5 Baseshots=3-shot, metric_mode=pass@1, n=32, sanitization=EvalPlus sanitized2026.06 | 76.69 | |
| Kimi-K2 Baseshots=3-shot, metric_mode=pass@1, n=32, sanitization=EvalPlus sanitized2026.06 | 72.14 | |
| DeepSeek-V3.2 Exp-Baseshots=3-shot, metric_mode=pass@1, n=32, sanitization=EvalPlus sanitized2026.06 | 58.66 |