Competitive Programming on Hard Bench (HB-15) pass@5 v2
47Correlation ScoreReflexion
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ReflexionBackbone=Haiku-4-5, Evaluation Protocol=pass@52026.06 | 47 | 86 | |
| AlgoSkill (MCTS)Backbone=Haiku-4-5, Variant=MCTS, Evaluation Protocol=pass@52026.06 | 47 | 100 | |
| CoTBackbone=Sonnet-4-5, Evaluation Protocol=pass@52026.06 | 40 | 83 | |
| AlgoSkill-GBackbone=Sonnet-4-5, Variant=Greedy, Evaluation Protocol=pass@52026.06 | 40 | 83 | |
| CoTBackbone=Haiku, Evaluation Protocol=pass@52026.06 | 27 | 100 | |
| AlgoSkill-GBackbone=Haiku, Variant=Greedy, Evaluation Protocol=pass@52026.06 | 27 | 100 | |
| DirectBackbone=Sonnet-4-5, Evaluation Protocol=pass@52026.06 | 27 | 0 | |
| Self-RefineBackbone=Haiku-4-5, Evaluation Protocol=pass@52026.06 | 27 | — | |
| DirectBackbone=Haiku, Evaluation Protocol=pass@52026.06 | 20 | 100 |