Software Vulnerability Detection on OSV-Bench 1.0 (test) benchmark leaderboard · SOTA2 Research

Software Vulnerability Detection on OSV-Bench 1.0 (test)

97.18Pointer Errors (Pass@1)Claude Opus 4.6

Evaluation Results

Method	Links
Claude Opus 4.62026.04		97.18	95.54	94.59	98.15	96.3	97.96	96.73
Claude Sonnet 42026.04		77.46	66.96	75.68	79.63	70.37	85.71	75.1
Claude Haiku 4.52026.04		76.06	66.07	70.27	75.93	72.22	83.67	76.33
Devstral 22026.04		74.65	72.32	75.68	85.19	77.78	85.71	80
Claude Opus 4.62026.04		70.42	72.32	60.81	62.96	69.44	79.59	74.69
Nova Premier2026.04		56.34	53.57	52.7	37.04	57.41	63.27	58.78
Claude Sonnet 42026.04		54.93	51.79	52.7	46.3	55.56	75.51	61.22
Qwen32026.04		54.93	37.5	44.59	38.89	47.22	55.1	49.8
Llama 4 Maverick2026.04		52.11	48.21	39.19	46.3	57.41	59.18	55.1
Doubao-1.5-pro2026.04		50.7	48.21	45.95	40.74	52.78	63.27	55.1
DeepSeek-V3.22026.04		47.89	53.57	51.35	59.26	52.78	63.27	56.73
Devstral 22026.04		46.48	41.96	47.3	42.59	48.15	51.02	48.16
Qwen32026.04		43.66	29.46	32.43	20.37	33.33	44.9	38.78
Claude Sonnet 3.52026.04		39.44	41.96	39.19	48.15	39.81	46.94	44.9
Claude Haiku 4.52026.04		39.44	41.96	41.89	42.59	43.52	59.18	50.2
Nova Premier2026.04		39.44	35.71	37.84	27.78	40.74	48.98	43.67
DeepSeek-Chat2026.04		38.02	39.29	36.49	44.44	43.52	51.02	46.53
DeepSeek-R1†2026.04		35.21	33.93	27.03	44.44	37.04	53.06	40.41
Llama 4 Maverick2026.04		33.8	33.04	25.68	35.19	40.74	42.86	38.78
DeepSeek-V3.22026.04		30.99	36.61	31.08	29.63	34.26	38.78	38.78
DeepSeek-R1†2026.04		21.13	22.32	18.92	29.63	28.7	24.49	26.53