Coding on MBPP (Solve Rate)
83.28Solve RateCapFlow
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CapFlowType=Learning, Setting=Multi-domain, Passes=12026.02 | 83.28 | — | |
| AFlowType=Refinement, Setting=Single-domain, Iterations=202026.02 | 82.99 | — | |
| ScoreFlowType=Learning, Setting=Single-domain, Iterations=52026.02 | 82.69 | — | |
| CapFlowType=Learning, Setting=Cross-domain, Passes=1, Cross-domain Domain Mapping=Reason,Math->Code2026.02 | 82.11 | — | |
| ScoreFlowType=Learning, Setting=Multi-domain, Iterations=52026.02 | 81.23 | — | |
| CoT-SCType=Manual, Setting=Manual2026.02 | 72.72 | — | |
| SPPType=Manual, Setting=Manual2026.02 | 72.72 | — | |
| CoTType=Manual, Setting=Manual2026.02 | 70.96 | — | |
| GPT-4o-miniType=Manual, Setting=Manual2026.02 | 70.67 | — | |
| ADASType=Refinement, Setting=Single-domain, Iterations=202026.02 | 70.08 | — | |
| Self-RefineType=Manual, Setting=Manual2026.02 | 69.5 | — | |
| Llama3.2Number of Parameters=1B, Instruction Fine-Tuning (IFT)=true, Evaluation Protocol=user-assistant conversations2026.03 | 39.6 | — | |
| MobileLLM-FlashNumber of Parameters=1.4B, Instruction Fine-Tuning (IFT)=true, Evaluation Protocol=user-assistant conversations2026.03 | 35.6 | — | |
| Gemma3Number of Parameters=1B, Instruction Fine-Tuning (IFT)=true, Evaluation Protocol=user-assistant conversations2026.03 | 35.2 | — | |
| MobileLLM-FlashNumber of Parameters=650M, Instruction Fine-Tuning (IFT)=true, Evaluation Protocol=user-assistant conversations2026.03 | 33 | — |