List your company
ResearchFormerly WizWand

Code Generation on CodeContests (Pass@1)

55.2Pass@1Oracle
-0.85613.69728.2542.803May 15, 2026May 16, 2026May 17, 2026May 18, 2026May 19, 2026May 20, 2026May 21, 2026
Updated 5mo ago

Evaluation results

MethodLinks
OracleModels=182026.0555.2
MARGIN c.Models=18, Confidence Source=consistency2026.0545.7
CAPSModel=Qwen3-4B-Thinking-2507, N=16, Temperature=0.62026.0544.8
MARGIN v.Models=18, Confidence Source=verbalized2026.0540.7
V1-InferModel=Qwen3-4B-Thinking-2507, N=16, Temperature=0.62026.0540.6
Best modelModels=182026.0540
CAPSModel=Qwen3-14B, N=16, Temperature=0.62026.0535.8
PointwiseModel=Qwen3-4B-Thinking-2507, N=16, Temperature=0.62026.0534.1
VanillaModel=Qwen3-4B-Thinking-2507, N=16, Temperature=0.62026.0531.4
V1-InferModel=Qwen3-14B, N=16, Temperature=0.62026.0530.9
V1-InferModel=GPT-OSS-20B, N=16, Temperature=0.62026.0530.9
CAPSModel=GPT-OSS-20B, N=16, Temperature=0.62026.0529.4
Raw verb.Models=182026.0527.3
PointwiseModel=Qwen3-14B, N=16, Temperature=0.62026.0524.5
PointwiseModel=GPT-OSS-20B, N=16, Temperature=0.62026.0524.5
VanillaModel=Qwen3-14B, N=16, Temperature=0.62026.0521.7
VanillaModel=GPT-OSS-20B, N=16, Temperature=0.62026.0521.4
CAPSModel=Qwen3-4B-Instruct-2507, N=16, Temperature=0.62026.0514.5
V1-InferModel=Qwen3-4B-Instruct-2507, N=16, Temperature=0.62026.057.9
PointwiseModel=Qwen3-4B-Instruct-2507, N=16, Temperature=0.62026.054.5
VanillaModel=Qwen3-4B-Instruct-2507, N=16, Temperature=0.62026.051.3