Refusal behavior analysis on Paired safety frame contrast items n=20 per model (test)
0.043Refusal Delta (eval vs dep)OLMo-3-Instruct (7B)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OLMo-3-Instruct (7B)Model family=OLMo-3, Model size (parameters)=7B, Protocol=paired-prompt protocol2026.05 | 0.043 | 0.141 | 0.075 | 0.088 | |
| Llama-3.1-70BModel family=Llama, Model size (parameters)=70B, Protocol=paired-prompt protocol2026.05 | -0.071 | 0.093 | 0.039 | 0.179 | |
| Phi-3.5-mini (3.8B)Model family=Phi, Model size (parameters)=3.8B, Protocol=paired-prompt protocol2026.05 | -0.089 | 0.054 | 0.057 | 0.139 | |
| Mistral-Small-3.2 (24B)Model family=Mistral, Model size (parameters)=24B, Protocol=paired-prompt protocol2026.05 | -0.15 | 0.003 | 0.114 | 0.02 | |
| Llama-3.1-8BModel family=Llama, Model size (parameters)=8B, Protocol=paired-prompt protocol2026.05 | -0.196 | 0.001 | 0.246 | 0.001 |