List your company
ResearchFormerly WizWand

Code Generation on LiveCodeBench October 2024 to January 2025 (test)

46.1Pass@1ThinkCoder
21.55627.92834.340.672Dec 30, 2024
Updated 8mo ago

Evaluation results

MethodLinks
ThinkCoderBackbone=GPT-4-Turbo, Self-refinement budget (n)=22024.1246.1
ThinkCoderBackbone=GPT-4-Turbo, Self-refinement budget (n)=52024.1244.9
ThinkCoderBackbone=GPT-4-Turbo, Self-refinement budget (n)=12024.1243.7
ThinkCoderBackbone=GPT-4o, Self-refinement budget (n)=52024.1241.3
ThinkCoderBackbone=GPT-4o, Self-refinement budget (n)=22024.1238.3
ThinkCoderBackbone=GPT-4o, Self-refinement budget (n)=12024.1235.3
ThinkCoderBackbone=GPT-4-Turbo, Self-refinement budget (n)=02024.1233.4
ThinkCoderBackbone=GPT-4o, Self-refinement budget (n)=02024.1222.5