Code Generation on SimuPhy benchmark leaderboard · SOTA2 Research

Code Generation on SimuPhy

99.9Avg Executable Rate @8 (E.R.)DeepSeek-V3.1

Evaluation Results

Method	Links
DeepSeek-V3.12026.02		99.9	95.2	64	14.5	100	99.4	90.4	40.7
GPT-oss-120b2026.02		99.8	93.7	76	14	100	99.7	91.9	37.4
Qwen3-32B2026.02		99.8	96.6	51	11.1	100	100	82.6	30.5
Qwen3-235B-A22B2026.02		99.7	95.9	68.5	15.1	100	100	91.3	41
DeepSeek-R1-05282026.02		99.7	97.3	77	21.5	100	100	93.4	52.7
Gemini-2.5-pro2026.02		99.1	77.3	38	12.7	100	99.7	77.8	37.4
GPT-oss-20b2026.02		99.1	92	61.6	10.5	100	100	85.6	32
GPT-o32026.02		98.9	75.2	64.1	15.9	100	100	95.5	45.2
GPT-5-medium2026.02		97.2	68.9	55.7	20.5	100	100	93.1	59.9
GPT-o4-mini2026.02		88.1	86.9	78.6	17.2	100	100	95.5	42.5