Constraint Following on ConstraintRot (test)
0Violation RateDeepSeek-V4 (control)
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek-V4 (control)Model=DeepSeek-V4, Condition=control (full ctx)2026.06 | 0 | |
| DeepSeek-V4 (defense_pin)Model=DeepSeek-V4, Condition=defense_pin2026.06 | 0 | |
| DeepSeek-V4 (defense_pin_attack)Model=DeepSeek-V4, Condition=defense_pin_attack2026.06 | 0 | |
| GLM-5.1 (control)Model=GLM-5.1, Condition=control (full ctx)2026.06 | 0 | |
| GLM-5.1 (compaction)Model=GLM-5.1, Condition=compaction2026.06 | 0 | |
| GLM-5.1 (defense_pin)Model=GLM-5.1, Condition=defense_pin2026.06 | 0 | |
| GLM-5.1 (defense_pin_attack)Model=GLM-5.1, Condition=defense_pin_attack2026.06 | 0 | |
| Qwen3.6 (control)Model=Qwen3.6, Condition=control (full ctx)2026.06 | 0 | |
| Qwen3.6 (defense_pin)Model=Qwen3.6, Condition=defense_pin2026.06 | 0 | |
| Qwen3.6 (defense_pin_attack)Model=Qwen3.6, Condition=defense_pin_attack2026.06 | 0 | |
| Kimi-K2.5 (control)Model=Kimi-K2.5, Condition=control (full ctx)2026.06 | 0 | |
| Kimi-K2.5 (defense_pin)Model=Kimi-K2.5, Condition=defense_pin2026.06 | 0 | |
| Kimi-K2.5 (defense_pin_attack)Model=Kimi-K2.5, Condition=defense_pin_attack2026.06 | 0 | |
| Claude-4.6 (control)Model=Claude-4.6, Condition=control (full ctx)2026.06 | 0 | |
| Claude-4.6 (attack_volume)Model=Claude-4.6, Condition=attack_volume2026.06 | 0 | |
| Claude-4.6 (attack_inject)Model=Claude-4.6, Condition=attack_inject2026.06 | 0 | |
| Claude-4.6 (defense_pin)Model=Claude-4.6, Condition=defense_pin2026.06 | 0 | |
| Claude-4.6 (defense_pin_attack)Model=Claude-4.6, Condition=defense_pin_attack2026.06 | 0 | |
| GPT-5.4m (control)Model=GPT-5.4m, Condition=control (full ctx)2026.06 | 0 | |
| GPT-5.4m (defense_pin)Model=GPT-5.4m, Condition=defense_pin2026.06 | 0 | |
| GPT-5.4m (defense_pin_attack)Model=GPT-5.4m, Condition=defense_pin_attack2026.06 | 0 | |
| Gemini-3.5f (control)Model=Gemini-3.5f, Condition=control (full ctx)2026.06 | 0 | |
| Gemini-3.5f (attack_inject)Model=Gemini-3.5f, Condition=attack_inject2026.06 | 0 | |
| Gemini-3.5f (defense_pin)Model=Gemini-3.5f, Condition=defense_pin2026.06 | 0 | |
| Gemini-3.5f (defense_pin_attack)Model=Gemini-3.5f, Condition=defense_pin_attack2026.06 | 0 | |
| Gemini-3.5f (compaction)Model=Gemini-3.5f, Condition=compaction2026.06 | 4 | |
| GLM-5.1 (attack_volume)Model=GLM-5.1, Condition=attack_volume2026.06 | 7 | |
| Claude-4.6 (compaction)Model=Claude-4.6, Condition=compaction2026.06 | 19 | |
| Gemini-3.5f (attack_volume)Model=Gemini-3.5f, Condition=attack_volume2026.06 | 19 | |
| GLM-5.1 (attack_inject)Model=GLM-5.1, Condition=attack_inject2026.06 | 22 | |
| Qwen3.6 (attack_volume)Model=Qwen3.6, Condition=attack_volume2026.06 | 26 | |
| Qwen3.6 (compaction)Model=Qwen3.6, Condition=compaction2026.06 | 30 | |
| Qwen3.6 (attack_inject)Model=Qwen3.6, Condition=attack_inject2026.06 | 33 | |
| Claude-4.6 (floor)Model=Claude-4.6, Condition=floor (no policy)2026.06 | 33 | |
| DeepSeek-V4 (floor)Model=DeepSeek-V4, Condition=floor (no policy)2026.06 | 37 | |
| GPT-5.4m (attack_volume)Model=GPT-5.4m, Condition=attack_volume2026.06 | 37 | |
| GPT-5.4m (attack_inject)Model=GPT-5.4m, Condition=attack_inject2026.06 | 37 | |
| Kimi-K2.5 (attack_inject)Model=Kimi-K2.5, Condition=attack_inject2026.06 | 41 | |
| GPT-5.4m (compaction)Model=GPT-5.4m, Condition=compaction2026.06 | 41 | |
| GLM-5.1 (floor)Model=GLM-5.1, Condition=floor (no policy)2026.06 | 44 | |
| Kimi-K2.5 (attack_volume)Model=Kimi-K2.5, Condition=attack_volume2026.06 | 44 | |
| DeepSeek-V4 (attack_volume)Model=DeepSeek-V4, Condition=attack_volume2026.06 | 48 | |
| Qwen3.6 (floor)Model=Qwen3.6, Condition=floor (no policy)2026.06 | 48 | |
| Kimi-K2.5 (floor)Model=Kimi-K2.5, Condition=floor (no policy)2026.06 | 48 | |
| GPT-5.4m (floor)Model=GPT-5.4m, Condition=floor (no policy)2026.06 | 56 | |
| DeepSeek-V4 (compaction)Model=DeepSeek-V4, Condition=compaction2026.06 | 59 | |
| DeepSeek-V4 (attack_inject)Model=DeepSeek-V4, Condition=attack_inject2026.06 | 59 | |
| Kimi-K2.5 (compaction)Model=Kimi-K2.5, Condition=compaction2026.06 | 59 | |
| Gemini-3.5f (floor)Model=Gemini-3.5f, Condition=floor (no policy)2026.06 | 63 |