Execution-based Output Prediction on CRUXEval-O (200-sample, seed 42, test) benchmark leaderboard · SOTA2 Research

Execution-based Output Prediction on CRUXEval-O (200-sample, seed 42, test)

55AccuracyQwen3-14B-Base

Evaluation Results

Method	Links
Qwen3-14B-Base2026.06		55
Qwen2.5-Coder-14B2026.06		54
Qwen2.5-14B2026.06		51
Qwen3-8B-Base2026.06		51
Qwen2.5-Coder-7B (anchor)2026.06		50.5
Qwen2.5-7B2026.06		49
Qwen3-4B-Base2026.06		45.5
DeepSeek-Coder-6.7B2026.06		45.5
Qwen2.5-Coder-1.5B2026.06		40
Qwen2.5-Coder-3B2026.06		39
CodeLlama-7B2026.06		36.5
Qwen3-1.7B-Base2026.06		35.5
Qwen2.5-3B2026.06		35
Qwen2.5-Coder-1.5B-Instruct2026.06		34
Qwen2.5-1.5B2026.06		33.5
Qwen3-0.6B-Base2026.06		27.5
Qwen2.5-Coder-0.5B-Instruct2026.06		27
Llama-2-7B2026.06		17.5
Qwen2.5-0.5B2026.06		7
Qwen2.5-Coder-0.5B2026.06		1