Coding Reasoning on Frontal Cortex
50Success RateCodeCytos
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CodeCytosBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 50 | 80.21 | — | — | — | |
| CodeCytosBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 48.67 | 73.17 | 77.32 | 77.41 | 71 | |
| CodeCytosBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 44.53 | 71.61 | 77.36 | 77.39 | 70 | |
| CodeCytosBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 38.37 | 72.83 | — | — | — | |
| CodeCytosBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 36.61 | 65.56 | 73.45 | 73.37 | 64 | |
| CodeCytosBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 34.8 | 66.12 | 74.12 | 74.25 | 64 | |
| CodeCytosBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 32.91 | 67.67 | 75.01 | 75.02 | 65 | |
| CodeCytosBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 29.8 | 66.65 | 75.07 | 74.95 | 64 | |
| CodeCytosBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=CodeCytos2026.05 | 24.9 | 60.39 | 74.18 | 74.27 | 59 | |
| CodeCytosBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=CodeCytos2026.05 | 21.35 | 63.51 | — | — | — | |
| CodeCytosBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=CodeCytos2026.05 | 18.13 | 53.53 | 65.27 | 65.25 | 51 | |
| CodeCytosBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 17.86 | 51.28 | 65.32 | 65.32 | 50 | |
| Tool-Augmented LLMBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 3.95 | 14.66 | 21.82 | 21.77 | 15 | |
| CodeCytosBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 3.63 | 15.77 | 27.42 | 27.42 | 17 | |
| Tool-Augmented LLMBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 2.6 | 10.53 | 16.13 | 16.15 | 11 | |
| Tool-Augmented LLMBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 2.35 | 10.52 | 17.63 | 17.81 | 11 | |
| CodeCytosBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=CodeCytos2026.05 | 2.18 | 10.35 | 19.35 | 19.35 | 11 | |
| Tool-Augmented LLMBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=Conventional Without CoT2026.05 | 1.86 | 8.04 | 12.88 | 12.86 | 8 | |
| Tool-Augmented LLMBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 1.61 | 7.22 | 12.86 | 12.93 | 8 | |
| Tool-Augmented LLMBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 1.42 | 6.42 | 11.3 | 11.34 | 7 | |
| Tool-Augmented LLMBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=With Zero-Shot CoT2026.05 | 1.38 | 4.06 | 6.68 | 12.9 | 7 | |
| Tool-Augmented LLMBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 1 | 4.3 | 7.34 | 7.22 | 5 | |
| Tool-Augmented LLMBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 0.97 | 2.8 | 4.48 | 8.06 | 5 | |
| Tool-Augmented LLMBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=With Few-Shot CoT2026.05 | 0.89 | 2.45 | 3.79 | 6.45 | 4 | |
| CodeCytosBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=CodeCytos + One Shot2026.05 | 0.89 | 4.42 | 8.8 | 8.81 | 5 | |
| Tool-Augmented LLMBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 0.81 | 2.26 | 3.49 | 5.65 | 4 | |
| CodeCytosBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=CodeCytos2026.05 | 0.75 | 3.72 | 7.27 | 7.29 | 4 | |
| Tool-Augmented LLMBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 0.65 | 1.94 | 3.23 | 6.45 | 4 | |
| Tool-Augmented LLMBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 0.64 | 3.08 | 5.71 | 5.66 | 3 | |
| Tool-Augmented LLMBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 0.56 | 2.5 | 4.02 | 4.01 | 3 | |
| Tool-Augmented LLMBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 0.48 | 1.45 | 2.42 | 4.84 | 3 | |
| Tool-Augmented LLMBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 0.32 | 0.97 | — | — | — | |
| CodeCytosBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=CodeCytos2026.05 | 0.24 | 1.21 | 2.44 | 2.4 | 1 | |
| CodeCytosBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 0.16 | 0.8 | 1.59 | 1.61 | 1 | |
| Tool-Augmented LLMBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 0.16 | 0.48 | — | — | — | |
| Tool-Augmented LLMBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 0.08 | 0.41 | — | — | — |