Developer Knowledge Evaluation on DevEval
61Win Rategpt-oss-120b (high)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| gpt-oss-120b (high)Params (B)=117, reasoning_effort=high, Reference model=Claude 4.5 Sonnet, Judge=gpt-oss-20b (high)2026.06 | 61 | 8.82 | 8.24 | |
| gpt-oss-20b (high)Params (B)=20, reasoning_effort=high, Reference model=Claude 4.5 Sonnet, Judge=gpt-oss-20b (high)2026.06 | 39 | 8.36 | 8.48 | |
| GLM-5.1Params (B)=754, reasoning_effort=high, Reference model=Claude 4.5 Sonnet, Judge=gpt-oss-20b (high)2026.06 | 32 | 8.17 | 8.67 | |
| Qwen3.6-35B-A3BParams (B)=35, reasoning_effort=high, Reference model=Claude 4.5 Sonnet, Judge=gpt-oss-20b (high)2026.06 | 22 | 7.59 | 8.75 | |
| MiniMax-M2.5Params (B)=230, reasoning_effort=high, Reference model=Claude 4.5 Sonnet, Judge=gpt-oss-20b (high)2026.06 | 17 | 6.82 | 8.86 | |
| gemma-4-E2b-itParams (B)=5.1, reasoning_effort=high, Reference model=Claude 4.5 Sonnet, Judge=gpt-oss-20b (high)2026.06 | 3 | 5.99 | 9.11 | |
| granite-4.0-h-smallParams (B)=32, reasoning_effort=high, Reference model=Claude 4.5 Sonnet, Judge=gpt-oss-20b (high)2026.06 | 2 | 5.3 | 9.15 |