List your company
ResearchFormerly WizWand

Code Generation on APPS Intermediate

81.95Pass RateMC-Tree-Of-Agents
18.239634.779851.3267.8602Feb 18, 2025Apr 19, 2025Jun 18, 2025Aug 18, 2025Oct 17, 2025Dec 16, 2025Feb 15, 2026
Updated 8mo ago

Evaluation results

MethodLinks
MC-Tree-Of-AgentsStrategy=Pruning2025.0281.95—67
MC-Tree-Of-Agents2025.0279.42—63
MC-Tree-Of-AgentsStrategy=Refine2025.0278.85—62
RethinkMCTS2025.0274.35—49
SingleBackbone=Claude2025.0273.6—57
SingleBackbone=GPT4omini2025.0272.89—50
ToT2025.0263.49—33
LogitsCoderModel Category=Decoding-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0261.334—
RethinkMCTSModel Category=Search-guided Reasoning Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0258.8829—
MCTSModel Category=Search-guided Reasoning Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0256.6723—
Guided DecodingModel Category=Decoding-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0255.2725—
Contrastive DecodingModel Category=Decoding-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0254.6325—
self-playModel Category=Reflection-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0248.7621—
LDB2025.0246.78—22
LATS2025.0245.86—20
Reflexion2025.0245.58—21
LATSModel Category=Search-guided Reasoning Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0244.4221—
RAP2025.0243.32—14
ReflexionModel Category=Reflection-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0241.3320—
ZeroShot2025.0240.57—19
LDBModel Category=Reflection-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0237.6127—
RAPModel Category=Reflection-based Models, Backbone=Qwen2.5-14B-Instruct, Rollout Budget=202026.0236.3213—
Zero-shotModel Category=Base, Backbone=Qwen2.5-14B-Instruct, Evaluation Protocol=Zero-shot2026.0231.5712—
DareFinetune Method=Dare, Base Model=Meta-llama-3.1-instruct-8b2025.0226.66—5
SFT on allFinetune Method=SFT on all, Base Model=Meta-llama-3.1-instruct-8b2025.0226.4—3
SFT on cluster 0Finetune Method=SFT on cluster 0, Base Model=Meta-llama-3.1-instruct-8b2025.0224.23—3
SFT on cluster 2Finetune Method=SFT on cluster 2, Base Model=Meta-llama-3.1-instruct-8b2025.0223.97—3
TwinFinetune Method=Twin, Base Model=Meta-llama-3.1-instruct-8b2025.0223.85—5
DisenLoRAFinetune Method=DisenLoRA, Base Model=Meta-llama-3.1-instruct-8b2025.0223.11—3
TiesFinetune Method=Ties, Base Model=Meta-llama-3.1-instruct-8b2025.0223.06—4
w/o tuningFinetune Method=w/o tuning, Base Model=Meta-llama-3.1-instruct-8b2025.0220.72—4
SFT on cluster 1Finetune Method=SFT on cluster 1, Base Model=Meta-llama-3.1-instruct-8b2025.0220.69—4