Hidden Objective Detection Evaluation Set Dependency 1.0 (out-of-domain)

100Exit RateGemini-3-Flash

Evaluation results

Method
Gemini-3-Flash2026.05	100	—	100	100	33.33	56.64	93.75	87.84	96	100	85.28
Gemini-3-Flash2026.05	100	—	100	100	21.82	78.57	100	100	100	100	88.93
Gemini-3-Flash2026.05	100	—	100	100	45	84.96	100	100	100	100	92.22
Gemini-3-Flash2026.05	100	—	100	100	16.36	90.48	100	100	100	100	89.65
Gemini-1.5-Flash2026.05	100	89.61	100	100	18.45	94	94	100	100	100	—
Gemini-1.5-Flash2026.05	100	84.88	100	100	4.3	60	100	99.6	100	100	—
Gemini-1.5-Flash2026.05	100	92.3	100	100	30.73	100	100	100	100	100	—
CoT-Guard 1.5B2026.05	95.5	78.86	94.87	100	18.2	18.23	100	85.8	98.93	98.22	—
CoT-Guard 1.5B2026.05	95.43	73.01	84.62	98.44	1.73	20.86	99.12	86.15	72.55	98.22	—
CoT-Guard 4B2026.05	88.89	—	79.17	88.64	32.05	53.1	80	89.95	80.69	90	75.83
CoT-Guard 4B2026.05	85.47	—	87.5	88.64	69.55	70.54	100	94.3	95.26	83.33	86.07
GPT-5.42026.05	84.6	—	100	100	25.4	12.4	87.5	55.4	94.67	81.36	71.26
Gemini-1.5-Flash2026.05	84.52	72.2	100	100	0.87	0.95	95.36	77.18	91.73	99.2	—
GPT-4o2026.05	83.9	56.86	100	100	0	4.8	84.4	22.6	52	64	—
Qwen2-1.5B2026.05	83.23	61.49	92.31	97.09	3.44	8.04	99.34	67.11	11.88	90.95	—
Qwen3-4B2026.05	81.91	—	91.67	90.91	31.52	46.84	100	92.95	73.38	91.75	77.88
Qwen3-4B2026.05	78.63	—	70.83	90.91	73.41	64.92	80	100	88.46	90	81.91
Qwen3-32B2026.05	77.78	—	100	98.28	41.67	21.24	100	87.42	94.67	83.84	78.32
Qwen3-8B2026.05	74.07	—	95.83	93.18	19.32	22.76	90	31.08	92	87.01	67.25
Qwen3-32B2026.05	65.38	—	100	96.04	28.64	12.39	93.75	38.01	58.67	49.15	60.23
Qwen3-8B2026.05	62.96	—	95.83	92.67	24.09	15.93	81.25	4.05	41.2	50.85	52.09
Qwen2-1.5B2026.05	60.08	65.4	87.18	99.61	1.74	11.36	98.2	35.1	97.74	97.63	—
Qwen3-4B2026.05	57.69	—	91.67	94.36	24.09	17.7	93.8	59.5	86.67	72.77	66.47
Qwen2-72B2026.05	51.61	40.89	97.44	88.24	0	0.75	90.73	5.81	20.41	12.99	—
GPT-5-mini2026.05	50	—	100	82.76	25	10.62	81.25	45.95	89.33	55.93	60.09
Qwen3-4B2026.05	44.44	—	91.27	93.81	20.45	7.96	93.75	29.73	38.62	35.59	50.62
GPT-4o-mini2026.05	40	50.7	100	84.38	0	2.82	64.2	24.1	73.33	67.47	—
Qwen3-4B2026.05	37.04	—	91.27	50	16.36	2.65	81.2	88.18	53.33	54.07	52.68
Qwen2-1.5B2026.05	32.9	37.58	100	98.24	0.86	1.68	93.38	2.06	1.6	7.47	—
Qwen2-7B2026.05	31.61	36.51	97.44	87.65	0	2.63	92.05	1.23	0.27	15.73	—
Qwen2-72B2026.05	25.81	57.26	94.87	89.65	3.48	1.5	88.6	46.5	81.92	82.97	—
Llama3.1-8B-Instruct2026.05	19.9	23.68	37.2	33	5	16.1	36.6	21.9	12.7	30.7	—
Llama3.1-8B-Instruct2026.05	19.23	—	10	6.35	14.77	3.71	30	35.94	20.23	40.34	20.06
GPT-5.42026.05	19.23	—	100	100	12.5	2.65	56.25	0	45.33	25.42	40.15
Qwen2-7B2026.05	17.42	44.15	94.87	84.96	0	2.63	72.2	2.1	70.13	53.07	—
Qwen2-1.5B2026.05	17.03	35.78	66.94	58.94	3.05	2.88	66.4	2.8	58.49	45.49	—
GPT-5-mini2026.05	11.54	—	100	60.34	12.5	1.77	43.75	2.7	37.33	15.25	31.69
Llama3.1-8B-Instruct2026.05	10.26	—	0	1.25	3.03	0.78	12.5	15.96	11.03	10.85	7.3
Qwen3-4B2026.05	7.69	—	80.95	29.31	16.67	0	62.5	82.77	6.67	8.47	32.78
GPT-4o-mini2026.05	7.1	20.92	78.95	24.71	0	0	76.16	0	0.79	0.53	—
Qwen2-1.5B2026.05	3.87	16.98	36.84	31.76	0	0.38	27.8	0.3	30.7	21.2	—
Qwen2-1.5B2026.05	3.23	14.56	42.11	21.6	0	0.19	63.67	0	0	0.27	—
Llama3.1-8B-Instruct2026.05	0.6	0.62	0	0	0	1.4	0.3	0	1.1	2.2	—
GPT-4o2026.05	0	29.5	100	84.21	0	0	80	0	0	1.33	—