Rumor Victim Susceptibility on CoPHEME

52.6ASR (Charlie Hebdo)Claude-4.5-Haiku

Evaluation results

Method
Claude-4.5-Haiku2026.01	52.6	0.74	20.6	34.5	0.7	10.3	32	0.71	8	50.9	0.74	14.6	63.6	0.77	34.5	16.7	0.61	16.7	42.4
Qwen2.5-3B-Inst2026.01	54.7	0.88	51.2	59.4	0.88	53.1	52.3	0.88	49.2	64.4	0.88	60	45.3	0.9	43.4	31	0.78	20.7	55.4
Qwen2.5-7B-Inst2026.01	67.8	0.86	62.1	82.4	0.84	68.5	62	0.85	56.5	64	0.86	54.3	65.5	0.86	61.8	36.7	0.8	26.7	67.1
Open-Weights Avg.2026.01	69.3	0.85	58.9	80.4	0.83	65.2	64.6	0.83	54	76	0.85	64.6	61.4	0.86	53.3	54.5	0.76	33.6	70.6
DS-R1-Distill-Qwen-1.5B2026.01	71	0.76	50.8	81	0.76	55.6	61.1	0.7	37.6	72.5	0.74	47.5	79.1	0.73	51.2	75	0.62	33.3	71.6
Qwen2.5-14B-Inst2026.01	71.4	0.85	59.5	81.3	0.82	62.6	60.2	0.83	46.9	85.9	0.85	69.3	53.9	0.89	50	69	0.75	27.6	71.9
DS-R1-Distill-Qwen-7B2026.01	74.9	0.88	66.3	92.1	0.85	75.2	76	0.87	69.5	81.2	0.89	74.5	66	0.92	60.4	66.7	0.86	60	79.2
Claude-3.5-Haiku2026.01	76.9	0.81	47.9	76.9	0.79	45	77.1	0.79	46.9	80.5	0.77	38.4	61.8	0.82	38.2	74.1	0.71	18.5	76.7
DS-R1-Distill-Qwen-14B2026.01	77	0.86	64.9	83.6	0.85	71.5	74.9	0.84	64.3	88.5	0.88	84.2	54.5	0.91	52.7	36.7	0.74	20	76.8
GPT-4.12026.01	77.1	0.88	73.7	64.8	0.88	60.6	60	0.87	57.5	77	0.89	74.5	54.5	0.94	54.5	16.7	0.9	16.7	65.9
Proprietary Avg.2026.01	78	0.82	60.1	78.8	0.8	55.7	68.3	0.8	46.2	81.4	0.82	61.2	64.7	0.85	53.3	51.8	0.76	23.6	74.4
GPT-4o2026.01	79.4	0.85	66.9	88.5	0.83	69.1	67	0.83	53	85.5	0.85	74.5	56.4	0.89	52.7	63.3	0.75	26.7	77.4
GPT-4.1-mini2026.01	79.9	0.85	68.4	80.6	0.82	50.9	68	0.81	42.5	77.6	0.87	67.9	63.6	0.91	63.6	20	0.81	16.7	72.7
GPT-4o-mini2026.01	81.7	0.83	67.4	92.1	0.82	70.3	79.5	0.81	59	86.7	0.85	78.2	74.5	0.85	60	66.7	0.82	53.3	83.1
GPT-4.1-nano2026.01	81.7	0.81	66.9	94.5	0.8	67.1	79.1	0.79	53.1	94.5	0.81	74.8	74.5	0.83	61.8	70	0.75	13.3	85.5
Claude-3-Haiku2026.01	94.8	0.8	69	98.8	0.79	72	83.5	0.76	50	98.8	0.79	66.9	68.5	0.82	61.1	86.7	0.71	26.7	91.5