Auto-coding on RWT 40 coding tasks 1.0 (test)
97.3Average Gradegrok-4-1-fast-non-reasoning
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| grok-4-1-fast-non-reasoningn=40, Code Base (tokens)=195,451, TREEFRAG (tokens)=8,3332026.01 | 97.3 | 2.11 | 5.2 | 0.2 | — | |
| claude-haiku-4-5n=40, Code Base (tokens)=210,519, TREEFRAG (tokens)=11,4742026.01 | 96.7 | 2.05 | 9.1 | 1.82 | — | |
| gpt-5.2n=40, Code Base (tokens)=196,361, TREEFRAG (tokens)=8,2552026.01 | 96.4 | 1.59 | 0.01 | 3.44 | — | |
| claude-sonnet-4-5n=40, Code Base (tokens)=210,519, TREEFRAG (tokens)=11,4742026.01 | 96.4 | 1.78 | 17.52 | 5.13 | — | |
| grok-4-1-fast-reasoningn=40, Code Base (tokens)=195,451, TREEFRAG (tokens)=8,2392026.01 | 96.3 | 1.93 | 16.92 | 0.2 | — | |
| claude-opus-4-5n=40, Code Base (tokens)=210,519, TREEFRAG (tokens)=11,4702026.01 | 96.1 | 1.8 | 16.63 | 8.61 | — | |
| gemini-2.5-pron=40, Code Base (tokens)=239,154, TREEFRAG (tokens)=12,1392026.01 | 95.6 | 1.64 | 32.54 | 2.79 | — | |
| grok-code-fast-1n=40, Code Base (tokens)=195,451, TREEFRAG (tokens)=8,3752026.01 | 95.4 | 2.77 | 18.22 | 0.28 | — | |
| gpt-4.1n=40, Code Base (tokens)=196,361, TREEFRAG (tokens)=8,2552026.01 | 94.9 | 2.85 | 23.27 | 2.61 | — | |
| gemini-2.5-flashn=40, Code Base (tokens)=239,154, TREEFRAG (tokens)=12,1402026.01 | 94.9 | 1.85 | 23.55 | 0.77 | — | |
| gpt-5.1n=40, Code Base (tokens)=196,361, TREEFRAG (tokens)=8,2552026.01 | 94.7 | 1.56 | 0.01 | 2.51 | — | |
| gemini-3-pro-previewn=40, Code Base (tokens)=239,154, TREEFRAG (tokens)=12,1352026.01 | 94.5 | 2.82 | 29.45 | 3.64 | — |