Coding Reasoning on NSCLC
63.1Success RateCodeCytos
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CodeCytosBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 63.1 | 82.45 | 85.81 | 85.83 | 81 | |
| CodeCytosBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 61.55 | 86.98 | — | — | — | |
| CodeCytosBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 60.46 | 82.35 | 88.1 | 88.05 | 81 | |
| CodeCytosBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 54.28 | 83.24 | 87.26 | 87.27 | 81 | |
| CodeCytosBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 52.93 | 81.55 | 86.56 | 86.53 | 80 | |
| CodeCytosBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 51.39 | 82.43 | — | — | — | |
| CodeCytosBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 46.61 | 78.07 | 83.6 | 83.58 | 76 | |
| CodeCytosBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 41.49 | 78.34 | 85.03 | 85.14 | 75 | |
| CodeCytosBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=CodeCytos2026.05 | 36.04 | 73.8 | 83.51 | 83.6 | 71 | |
| CodeCytosBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=CodeCytos2026.05 | 26.02 | 67.63 | 78.28 | 78.44 | 64 | |
| CodeCytosBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=CodeCytos + Few Shot2026.05 | 21.21 | 60.89 | 75.37 | 75.37 | 59 | |
| CodeCytosBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=CodeCytos2026.05 | 20.83 | 62.12 | — | — | — | |
| CodeCytosBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 8.04 | 31.34 | 47.76 | 47.76 | 32 | |
| Tool-Augmented LLMBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 6.07 | 19.62 | 29.85 | 29.89 | 20 | |
| Tool-Augmented LLMBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 5.97 | 20.34 | 30.58 | 30.63 | 20 | |
| Tool-Augmented LLMBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 5.69 | 19.3 | 29.93 | 29.87 | 20 | |
| Tool-Augmented LLMBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=Conventional Without CoT2026.05 | 4.03 | 14.04 | 22.42 | 22.42 | 14 | |
| CodeCytosBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=CodeCytos2026.05 | 3.66 | 13.76 | 23.13 | 23.13 | 14 | |
| Tool-Augmented LLMBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=With Zero-Shot CoT2026.05 | 3.36 | 8.33 | 12.27 | 20.15 | 13 | |
| Tool-Augmented LLMBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=With Few-Shot CoT2026.05 | 3.13 | 6.97 | 10.1 | 16.42 | 10 | |
| Tool-Augmented LLMBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 2.45 | 9.89 | 16.44 | 16.42 | 10 | |
| Tool-Augmented LLMBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 2.39 | 8.66 | 13.4 | 13.43 | 9 | |
| Tool-Augmented LLMBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 2.17 | 8.98 | 14.94 | 14.93 | 9 | |
| Tool-Augmented LLMBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 2.09 | 5.7 | 8.75 | 14.93 | 9 | |
| Tool-Augmented LLMBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 2.01 | 4.33 | 6.18 | 10.45 | 7 | |
| Tool-Augmented LLMBackbone=Qwen3-Coder-30B-A3B-Instruct, Backbone Size=30.5 Bilions Parameters, Pretrain Data=36 trillions of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 1.94 | 7.25 | 12.68 | 12.71 | 8 | |
| CodeCytosBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=CodeCytos + One Shot2026.05 | 1.91 | 5.8 | 9.68 | 9.73 | 6 | |
| Tool-Augmented LLMBackbone=Facebook CWM, Backbone Size=32 Bilions Parameters, Pretrain Data=13 trillions of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 1.72 | 3.91 | 5.6 | 9.7 | 6 | |
| Tool-Augmented LLMBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 1.64 | 6.05 | 9.75 | 9.75 | 6 | |
| CodeCytosBackbone=Devstral-Small-2505, Backbone Size=24 Bilions Parameters, Pretrain Data=undisclosed, Evaluation Protocol=CodeCytos2026.05 | 1.55 | 3.93 | 6.03 | 5.97 | 4 | |
| Tool-Augmented LLMBackbone=moonshotai/Kimi-Linear-48B-A3B-Instruct, Backbone Size=48 Billions Parameters, Pretrain Data=5.7 trillions of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 1.34 | 3.74 | 5.78 | 9.7 | 6 | |
| CodeCytosBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=CodeCytos2026.05 | 0.6 | 2.8 | 5.2 | 5.19 | 3 | |
| CodeCytosBackbone=Qwen2.5-Coder-32B-Instruct, Backbone Size=32.5 Billions Parameters, Pretrain Data=5.5 trillion of training tokens, Evaluation Protocol=CodeCytos + One Shot2026.05 | 0.45 | 2.26 | 4.51 | 4.43 | 2 | |
| Tool-Augmented LLMBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=Conventional Without CoT2026.05 | 0.45 | 1.79 | — | — | — | |
| Tool-Augmented LLMBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=With Zero-Shot CoT2026.05 | 0.45 | 1.2 | — | — | — | |
| Tool-Augmented LLMBackbone=GLM-4.5 Air, Backbone Size=106 Billions Parameters, Pretrain Data=23 trillions of training tokens, Evaluation Protocol=With Few-Shot CoT2026.05 | 0.45 | 1.34 | — | — | — |