URL Vulnerability Evaluation on MalURLBench 1.0 (test) benchmark leaderboard · SOTA2 Research

URL Vulnerability Evaluation on MalURLBench 1.0 (test)

100Success Rate (Commercial)Mixtral-8x7b

Evaluation Results

Method	Links
Mixtral-8x7b2026.01		100	100	100	100	100	100	100	100	100	100
Llama2-7B-chat-hf2026.01		100	94	98	100	94	99	69	96	99	100
Mistral-small2026.01		98	94	100	100	98	97	100	95	100	100
GPT-4o-mini2026.01		97	66	100	100	100	91	87	64	100	100
Deepseek-chat2026.01		92	78	100	75	50	81	90	88	97	94
Llama-3-8b2026.01		72	58	53	100	10	32	33	7	10	95
Llama-3-8b2026.01		72	58	53	100	20	32	33	33	10	95
Mistral-7b2026.01		70	46	63	60	38	16	52	48	53	65
Deepseek-coder2026.01		33	6	1	72	57	17	8	15	46	85
Deepseek-coder2026.01		33	33	27	72	57	17	32	15	46	85
GPT-3.5-turbo2026.01		23	39	40	41	20	13	27	34	42	50
Llama-3-70b2026.01		21	41	34	66	26	34	26	47	21	41
GPT-4o2026.01		18	13	5	74	69	29	37	1	22	77
GPT-4o2026.01		18	13	12	74	69	29	37	31	22	77
URLGuard2026.01		14	5	1	61	0	0	—	0	0	70
Qwen-plus2026.01		12	22	23	51	25	68	12	20	51	73
Llama-3-70b2026.01		3	2	34	66	26	5	26	7	21	41
GPT-3.5-turbo2026.01		1	6	4	41	9	13	9	0	8	50
Qwen-plus2026.01		1	0	23	51	1	68	12	20	51	73