Security Goal Extraction on SecGoal (test) benchmark leaderboard · SOTA2 Research

Security Goal Extraction on SecGoal (test)

100RecallGemma2-9b-FT

Evaluation Results

Method	Links
Gemma2-9b-FT2026.04		100	3	3	5	0	0	100	100
Gemma2-9b-FT2026.04		100	7	5	5	0	2	71.4	83.3
Gemma2-9b-FT2026.04		100	17	11	8	0	6	64.7	78.6
Gemma2-9b-FT2026.04		100	34	20	6	0	14	58.8	74.1
Qwen2.5-7b-FT2026.04		100	5	4	5	0	1	80	88.9
Qwen2.5-7b-FT2026.04		100	7	6	5	0	1	85.7	92.3
DeepSeek-V3.22026.04		100	26	6	5	0	20	23.1	37.5
DeepSeek-V3.22026.04		100	21	6	5	0	15	28.6	44.4
DeepSeek-V3.22026.04		100	375	20	8	0	355	5.3	10.1
DeepSeek-V3.22026.04		100	310	43	6	0	267	13.9	24.4
Gemini2.5-Flash2026.04		100	28	6	5	0	22	21.4	35.3
Gemini2.5-Flash2026.04		100	19	5	5	0	14	26.3	41.7
Gemini2.5-Flash2026.04		100	397	21	8	0	377	5.3	10
Gemini2.5-Flash2026.04		100	365	35	6	0	330	9.6	17.5
DeepSeek-V3.2Thinking2026.04		100	24	6	5	0	18	25	40
DeepSeek-V3.2Thinking2026.04		100	19	5	5	0	14	26.3	41.7
DeepSeek-V3.2Thinking2026.04		100	430	27	8	0	403	6.3	11.8
DeepSeek-V3.2Thinking2026.04		100	356	37	6	0	319	10.4	18.8
Gemini2.5-Flash2026.04		97.1	—	—	—	—	—	16	26.7
DeepSeek-V3.2Thinking2026.04		97.1	—	—	—	—	—	17.7	29
Gemma2-9b-FT2026.04		94.3	—	—	—	—	—	73.6	81.7
DeepSeek-V3.22026.04		94.3	—	—	—	—	—	18.9	30.3
Qwen2.5-7b-FT2026.04		91.3	—	—	—	—	—	72.3	80.4
Qwen2.5-7b-FT2026.04		87.5	15	8	7	1	7	53.3	66.3
Qwen2.5-7b-FT2026.04		85.7	30	22	12	2	8	73.3	79
Gemini2.5-Flash2026.04		85.7	161	28	12	2	133	17.4	28.9
DeepSeek-V3.2Thinking2026.04		85.7	153	31	12	2	122	20.3	32.8
Qwen2.5-7b-FT2026.04		83.3	29	20	5	1	9	69	75.5
Gemma2-9b-FT2026.04		71.4	31	22	10	4	9	71	71.2
DeepSeek-V3.22026.04		71.4	137	32	10	4	105	23.4	35.2