Jailbreak Attack on AUTORISE Frontier
52ASRDeepSeek-V3.2†
Evaluation results
| Method | Links | |
|---|---|---|
| DeepSeek-V3.2†Model Category=Standard, Evaluation Protocol=Three-judge majority vote, Development Status=AR-Standard dev target, Independent evaluation runs=32026.04 | 52 | |
| Grok 3†Model Category=Standard, Evaluation Protocol=Three-judge majority vote, Development Status=AR-Standard dev target, Independent evaluation runs=32026.04 | 44.5 | |
| Grok 4Model Category=Frontier, Evaluation Protocol=Three-judge majority vote, Development Status=Held-out, Independent evaluation runs=32026.04 | 44.5 | |
| GPT-4.1-miniModel Category=Standard, Evaluation Protocol=Three-judge majority vote, Development Status=Held-out, Independent evaluation runs=32026.04 | 43.5 | |
| GPT-4.1†Model Category=Standard, Evaluation Protocol=Three-judge majority vote, Development Status=AR-Standard dev target, Independent evaluation runs=32026.04 | 42 | |
| Phi-4-miniModel Category=Standard, Evaluation Protocol=Three-judge majority vote, Development Status=Held-out, Independent evaluation runs=32026.04 | 41.5 | |
| GPT-4o†Model Category=Standard, Evaluation Protocol=Three-judge majority vote, Development Status=AR-Standard dev target, Independent evaluation runs=32026.04 | 40.5 | |
| Kimi-K2.5Model Category=Standard, Evaluation Protocol=Three-judge majority vote, Development Status=Held-out, Independent evaluation runs=32026.04 | 40.5 | |
| o4-mini*Model Category=Frontier, Evaluation Protocol=Three-judge majority vote, Development Status=AR-Frontier dev target, Independent evaluation runs=32026.04 | 26.8 | |
| GPT-5.4*Model Category=Frontier, Evaluation Protocol=Three-judge majority vote, Development Status=AR-Frontier dev target, Independent evaluation runs=32026.04 | 12.2 | |
| GPT-5.2Model Category=Frontier, Evaluation Protocol=Three-judge majority vote, Development Status=Held-out, Independent evaluation runs=32026.04 | 8.5 |