Hidden Objective Monitoring on Prompt-Manipulation A1 Standard benchmark leaderboard · SOTA2 Research

Hidden Objective Monitoring on Prompt-Manipulation A1 Standard

100Exit Success RateGemini-1.5-Flash

Evaluation Results

Method	Links
Gemini-1.5-Flash2026.05		100	100	100	24.22	93.2	100	100	100	100	90.82
Gemini-1.5-Flash2026.05		100	100	100	0	59.26	96.8	98.3	100	100	83.82
Gemini-1.5-Flash2026.05		100	100	100	42.25	99.07	96.84	100	100	100	93.13
CoT-Guard 4B2026.05		95.79	94.87	100	0.93	23.97	100	68.6	98.9	98.2	75.7
Gemini-1.5-Flash2026.05		92.9	100	100	0	74.2	94.3	74.22	94.65	98.93	81.02
CoT-Guard 4B2026.05		92.6	84.6	98.4	2.8	15.2	99.12	74.7	67.5	98.2	70.35
GPT-4o2026.05		82.1	100	100	0	0	83.3	13.8	77.33	52	56.5
Qwen2-1.5B2026.05		81.4	92.3	97.7	1.9	5.6	98.1	72.1	11.1	92.7	61.43
Qwen2-1.5B2026.05		64.68	89.74	99.22	0.93	26.78	98.2	28.8	95.65	97.63	66.85
Qwen2-72B2026.05		55.71	97.44	87.27	0.93	0.37	82.28	2.09	13.5	14.84	39.38
Qwen2-1.5B2026.05		43.57	95.45	98.18	0	0.19	87.34	1.73	2.4	4.8	37.07
Qwen2-7B2026.05		40	97.4	88.5	0	0.7	83.54	1.03	0.27	14.13	36.17
GPT-4o-mini2026.05		34.29	95.45	77.27	0	0	44.9	8.7	67.73	48.8	41.9
Qwen2-72B2026.05		31.43	94.87	83.91	0	0.93	84.1	36.2	78.21	76.88	54.06
Qwen2-1.5B2026.05		19.57	65.03	55.38	0	2.25	62.3	1.3	57.42	40.13	33.71
Llama3.1-8B-Instruct2026.05		18.2	32.2	35.9	12.2	15.2	45.2	15.9	11	30.7	24.06
Qwen2-7B2026.05		15.71	94.87	82.1	0	1.12	49.4	1.4	59.2	52.6	39.6
GPT-4o-mini2026.05		7.86	72.73	17.58	0	0	67.09	0	0.26	0.53	18.45
GPT-4o2026.05		3.6	100	87.1	0	0	79.17	0	0	0	29.99
Qwen2-1.5B2026.05		3.2	42.1	21.6	0	0.2	47.05	0	0	1.6	12.86
Qwen2-1.5B2026.05		2.86	45.45	30.91	0	0	8.2	0.3	19.5	13.1	13.37
Llama3.1-8B-Instruct2026.05		0	0	0	1	0.2	0.6	0	1.5	2	0.59