coding-reasoning on Tonsil (test)
58.55Success RateCodeCytos + Few Shot
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CodeCytos + Few ShotBackbone=Qwen3-Coder-30B-A3B-Instruct, Framework=CodeAct-based, Prompting Strategy=Few Shot2026.05 | 58.55 | 81.6 | 87.12 | 88.53 | 82 | |
| CodeCytos + One ShotBackbone=Qwen3-Coder-30B-A3B-Instruct, Framework=CodeAct-based, Prompting Strategy=One Shot2026.05 | 52.25 | 78.3 | 83.75 | 84.75 | 78 | |
| CodeCytos + Few ShotBackbone=Qwen2.5-Coder-32B-Instruct, Framework=CodeAct-based, Prompting Strategy=Few Shot2026.05 | 47.65 | 76.88 | 82.52 | 83.17 | 76 | |
| CodeCytos + Few ShotBackbone=Facebook CWM, Framework=CodeAct-based, Prompting Strategy=Few Shot2026.05 | 44.44 | 77.15 | 82.89 | 83.15 | 76 | |
| CodeCytos + One ShotBackbone=Facebook CWM, Framework=CodeAct-based, Prompting Strategy=One Shot2026.05 | 42.62 | 76.26 | 82.71 | 83.23 | 75 | |
| CodeCytosBackbone=Facebook CWM, Framework=CodeAct-based, Prompting Strategy=Base2026.05 | 29.06 | 68.72 | 80.87 | 78.44 | 69 | |
| CodeCytosBackbone=Qwen3-Coder-30B-A3B-Instruct, Framework=CodeAct-based, Prompting Strategy=Base2026.05 | 24.68 | 61.44 | 72.92 | 74.72 | 62 | |
| Tool-Augmented LLM (Few-Shot CoT)Backbone=Qwen2.5-Coder-32B-Instruct, Framework=With, Prompting Strategy=Few-Shot CoT2026.05 | 7.17 | 23.22 | 33.13 | 35.96 | 25 | |
| Tool-Augmented LLM (Zero-Shot CoT)Backbone=Qwen2.5-Coder-32B-Instruct, Framework=With, Prompting Strategy=Zero-Shot CoT2026.05 | 5.62 | 20.51 | 32.12 | 35.82 | 23 | |
| Tool-Augmented LLM (Without CoT)Backbone=Qwen2.5-Coder-32B-Instruct, Framework=Conventional, Prompting Strategy=Without CoT2026.05 | 4.65 | 18.33 | 29.04 | 32.01 | 21 | |
| Tool-Augmented LLM (Few-Shot CoT)Backbone=Qwen3-Coder-30B-A3B-Instruct, Framework=With, Prompting Strategy=Few-Shot CoT2026.05 | 2.47 | 9.06 | 15.29 | 17.51 | 11 | |
| Tool-Augmented LLM (Without CoT)Backbone=Facebook CWM, Framework=Conventional, Prompting Strategy=Without CoT2026.05 | 2.05 | 7.54 | 13.24 | 15.34 | 9 | |
| Tool-Augmented LLM (Zero-Shot CoT)Backbone=Qwen3-Coder-30B-A3B-Instruct, Framework=With, Prompting Strategy=Zero-Shot CoT2026.05 | 1.7 | 6.22 | 10.06 | 11.46 | 7 | |
| Tool-Augmented LLM (Without CoT)Backbone=Qwen3-Coder-30B-A3B-Instruct, Framework=Conventional, Prompting Strategy=Without CoT2026.05 | 1.65 | 6.19 | 10.12 | 11.45 | 7 | |
| CodeCytos + One ShotBackbone=Qwen2.5-Coder-32B-Instruct, Framework=CodeAct-based, Prompting Strategy=One Shot2026.05 | 1.53 | 7.4 | 14.22 | 16.77 | 9 | |
| Tool-Augmented LLM (Zero-Shot CoT)Backbone=Facebook CWM, Framework=With, Prompting Strategy=Zero-Shot CoT2026.05 | 1.34 | 3.39 | 4.98 | 9.16 | 6 | |
| Tool-Augmented LLM (Few-Shot CoT)Backbone=Facebook CWM, Framework=With, Prompting Strategy=Few-Shot CoT2026.05 | 1.21 | 3.28 | 5.08 | 10.69 | 6 | |
| CodeCytosBackbone=Qwen2.5-Coder-32B-Instruct, Framework=CodeAct-based, Prompting Strategy=Base2026.05 | 0.89 | 4.45 | 8.91 | 10.7 | 6 |