Code Generation on LiveCodeBench v6, HumanEval+, MBPP+, and SciCode
0.992Pass@1DeepSeek-V3.1
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek-V3.1Thinking Mode=true, Parameters=685B-A37B, Lang=English, Sub-Dataset=MBPP+2026.01 | 0.992 | |
| GLM-4.6Thinking Mode=true, Parameters=357B-A32B, Lang=English, Sub-Dataset=MBPP+2026.01 | 0.989 | |
| DeepSeek-V3.1Thinking Mode=true, Parameters=685B-A37B, Lang=Korean, Sub-Dataset=HumanEval+ ko2026.01 | 0.939 | |
| A.X K1Thinking Mode=true, Parameters=519B-A33B, Lang=English, Sub-Dataset=MBPP+2026.01 | 0.93 | |
| A.X K1Thinking Mode=true, Parameters=519B-A33B, Lang=Korean, Sub-Dataset=HumanEval+ ko2026.01 | 0.902 | |
| A.X K1Thinking Mode=true, Parameters=519B-A33B, Lang=English, Sub-Dataset=HumanEval+2026.01 | 0.872 | |
| DeepSeek-V3.1Thinking Mode=true, Parameters=685B-A37B, Lang=English, Sub-Dataset=HumanEval+2026.01 | 0.86 | |
| GLM-4.6Thinking Mode=true, Parameters=357B-A32B, Lang=Korean, Sub-Dataset=HumanEval+ ko2026.01 | 0.86 | |
| GLM-4.6Thinking Mode=true, Parameters=357B-A32B, Lang=English, Sub-Dataset=HumanEval+2026.01 | 0.835 | |
| GLM-4.6Thinking Mode=true, Parameters=357B-A32B, Lang=English, Version=25.2.1 - 25.5.1, Sub-Dataset=LiveCodeBench v62026.01 | 0.76 | |
| A.X K1Thinking Mode=true, Parameters=519B-A33B, Lang=English, Version=25.2.1 - 25.5.1, Sub-Dataset=LiveCodeBench v62026.01 | 0.758 | |
| A.X K1Thinking Mode=true, Parameters=519B-A33B, Lang=Korean, Sub-Dataset=LiveCodeBench-ko2026.01 | 0.731 | |
| DeepSeek-V3.1Thinking Mode=true, Parameters=685B-A37B, Lang=English, Version=25.2.1 - 25.5.1, Sub-Dataset=LiveCodeBench v62026.01 | 0.695 | |
| DeepSeek-V3.1Thinking Mode=true, Parameters=685B-A37B, Lang=Korean, Sub-Dataset=LiveCodeBench-ko2026.01 | 0.662 | |
| GLM-4.6Thinking Mode=true, Parameters=357B-A32B, Lang=Korean, Sub-Dataset=LiveCodeBench-ko2026.01 | 0.559 | |
| DeepSeek-V3.1Thinking Mode=true, Parameters=685B-A37B, Lang=English, Sub-Dataset=SciCode2026.01 | 0.391 | |
| GLM-4.6Thinking Mode=true, Parameters=357B-A32B, Lang=English, Sub-Dataset=SciCode2026.01 | 0.384 | |
| A.X K1Thinking Mode=true, Parameters=519B-A33B, Lang=English, Sub-Dataset=SciCode2026.01 | 0.324 |