List your company
ResearchFormerly WizWand

Code Generation on CodeContests

89.09Pass@1Solvita
1.41824.17946.9469.701Aug 2, 2024Nov 18, 2024Mar 6, 2025Jun 23, 2025Oct 9, 2025Jan 25, 2026May 14, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SolvitaBackbone=DeepSeek V4 Pro2026.0589.09——
SolvitaBackbone=GPT-5.42026.0582.42——
Codex CLIBackbone=GPT-5.42026.0581.82——
SolvitaBackbone=Claude Opus 4.62026.0580.61——
Claude CodeBackbone=Claude Opus 4.62026.0580——
Codex CLIBackbone=DeepSeek V4 Pro2026.0579.39——
SolvitaBackbone=Grok2026.0578.18——
Claude CodeBackbone=DeepSeek V4 Pro2026.0576.97——
Codex CLIBackbone=Grok2026.0573.33——
Claude CodeBackbone=Grok2026.0572.12——
Claude CodeBackbone=GPT-5.42026.0570.91——
AlphaCodiumBackbone=DeepSeek V4 Pro2026.0570.91——
Codex CLIBackbone=Claude Opus 4.62026.0570.3——
SolvitaBackbone=Qwen3.62026.0569.7——
MapCoderBackbone=DeepSeek V4 Pro2026.0566.67——
AlphaCodiumBackbone=Claude Opus 4.62026.0564.24——
MapCoderBackbone=Grok2026.0562.42——
AlphaCodiumBackbone=GPT-5.42026.0560.61——
AlphaCodiumBackbone=Grok2026.0560.61——
MapCoderBackbone=Claude Opus 4.62026.0560——
Codex CLIBackbone=Qwen3.62026.0560——
PromptBridgeSource Model=GPT-4o, Target Model=o4-mini2025.1258.79——
Claude CodeBackbone=Qwen3.62026.0558.79——
GPT-5 OptimizerSource Model=GPT-4o, Target Model=o4-mini2025.1257.7——
MapCoderBackbone=GPT-5.42026.0557.58——
PromptBridgeSource Model=GPT-4o, Target Model=o32025.1256.36——
GEPASource Model=GPT-4o, Target Model=o4-mini2025.1255.15——
AlphaCodiumBackbone=Qwen3.62026.0553.33——
MapCoderBackbone=Qwen3.62026.0550.91——
Direct TransferSource Model=GPT-4o, Target Model=o32025.1248.61——
MIPROv2Source Model=GPT-4o, Target Model=o4-mini2025.1248.28——
Single-passBackbone=DeepSeek V4 Pro2026.0547.27——
GEPASource Model=GPT-4o, Target Model=o32025.1246.19——
MIPROv2Source Model=GPT-4o, Target Model=o32025.1245.65——
Single-passBackbone=Claude Opus 4.62026.0544.85——
Direct TransferSource Model=GPT-4o, Target Model=o4-mini2025.1240.12——
ThinkCoderBackbone=GPT-4o2024.1240.1——
Single-passBackbone=GPT-5.42026.0540——
Single-passBackbone=Grok2026.0538.18——
LPWBackbone=GPT-4o2024.1234.7——
Single-passBackbone=Qwen3.62026.0533.94——
GPT-5 OptimizerSource Model=GPT-4o, Target Model=o32025.1231.88——
LDBBackbone=GPT-4o2024.1229.3——
PromptBridgeSource Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.1223.84——
MIPROv2Source Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.1221.82——
Direct TransferSource Model=GPT-4o, Target Model=Gemma3-27B-it2025.1221.21——
Direct TransferSource Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.1220.2——
GPT-5 OptimizerSource Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.1219.8——
GPT-5 OptimizerSource Model=GPT-4o, Target Model=Qwen3-32B2025.1219.8——
GEPASource Model=GPT-4o, Target Model=Llama3.1-70B-Instruct2025.1218.99——
GPT-5 OptimizerSource Model=GPT-4o, Target Model=Gemma3-27B-it2025.1218.99——
PromptBridgeSource Model=GPT-4o, Target Model=Qwen3-32B2025.1218.79——
PromptBridgeSource Model=GPT-4o, Target Model=Gemma3-27B-it2025.1216.97——
ARCHCODECoT=true, Code Filtering=true, NFRs=true2024.0816.5216.6717.37
MPSCCoT=true, Code Filtering=true, NFRs=false, Base Model=GPT-3.5-Turbo2024.0814.3917.1617.76
Direct TransferSource Model=GPT-4o, Target Model=Qwen3-32B2025.1213.94——
ALGOCoT=true, Code Filtering=false, NFRs=false, Base Model=GPT-3.5-Turbo2024.081212—
CODETCoT=true, Code Filtering=true, NFRs=false, Base Model=GPT-3.5-Turbo2024.089.9210.1810.3
MBR-EXECCoT=true, Code Filtering=true, NFRs=false, Base Model=GPT-3.5-Turbo2024.088.258.879.08
PromptBridgeSource Model=GPT-4o, Target Model=Llama3.1-8B-Instruct2025.127.68——
Direct TransferSource Model=GPT-4o, Target Model=Llama3.1-8B-Instruct2025.127.47——
GPT-5 OptimizerSource Model=GPT-4o, Target Model=Llama3.1-8B-Instruct2025.127.27——
BRAINSTORMCoT=true, Code Filtering=false, NFRs=false, Base Model=GPT-3.5-Turbo2024.087—14.7
GPT-4CoT=false, Code Filtering=false, NFRs=false2024.086.078.2311.67
GPT-3.5-Turbo + CoTCoT=true, Code Filtering=false, NFRs=false2024.085.828.5713.53
GPT-4oModel Role=Source Model2025.125.45——
GPT-4oSource Model=GPT-4o, Evaluation Context=Source Performance Reference2025.125.45——
GPT-3.5-TurboCoT=false, Code Filtering=false, NFRs=false2024.084.797.0210.06