Security Evaluation on SecEval
92.3AccuracyGPT-5
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-5Number of trials=52026.01 | 92.3 | 0.002 | |
| GPT-4.1Number of trials=52026.01 | 91.9 | 0.001 | |
| GPT-5-MiniNumber of trials=52026.01 | 91.1 | 0.003 | |
| O3-MiniNumber of trials=52026.01 | 90.8 | 0.002 | |
| Llama-3.3-70B-InstructNumber of trials=52026.01 | 90.6 | 0.004 | |
| GPT-OSS-120BNumber of trials=52026.01 | 90.4 | 0.005 | |
| Phi-4Number of trials=52026.01 | 89.8 | 0.004 | |
| Qwen-3-14BNumber of trials=52026.01 | 89.2 | 0.003 | |
| Qwen-3-8BNumber of trials=52026.01 | 88.6 | 0.003 | |
| GPT-5-NanoNumber of trials=52026.01 | 88.4 | 0.005 | |
| Llama-Primus-Nemotron-70B-InstructNumber of trials=52026.01 | 88.2 | 0.005 | |
| GPT-OSS-20BNumber of trials=52026.01 | 87 | 0.006 | |
| Llama-3.3-70B-Ins-DAPSize [B]=70, Base Alignment=SFT + RLHF, Extra Tuning=DAP2025.06 | 86.4 | — | |
| Foundation-Sec-8B-ReasoningNumber of trials=52026.01 | 84.8 | 0.005 | |
| Llama-3.1-8B-InstructNumber of trials=52026.01 | 83.2 | 0.003 | |
| Foundation-Sec-8B-InstructNumber of trials=52026.01 | 82.9 | 0.003 | |
| Llama-Primus-MergedNumber of trials=52026.01 | 81.1 | 0.009 | |
| DeepHat-V1-7BNumber of trials=52026.01 | 79 | 0.005 | |
| DSR1D-Qwen-14B-DAPSize [B]=14, Base Alignment=KD + SFT, Extra Tuning=DAP2025.06 | 78.9 | — | |
| Llama-3.1-8B-DAPSize [B]=8, Base Alignment=None (pretrain only), Extra Tuning=DAP2025.06 | 74.8 | — | |
| CyberPal-2.0-20BParameters=20B2025.10 | 72.86 | — | |
| CyberPal-2.0-14BParameters=14B2025.10 | 69.71 | — | |
| gpt-oss-120BParameters=120B2025.10 | 68.02 | — | |
| gpt-oss-20BParameters=20B2025.10 | 67.65 | — | |
| CyberPal-2.0-8BParameters=8B2025.10 | 66.93 | — | |
| Llama-3-8B-BaseSize [B]=8, Base Alignment=None (pretrain only), Extra Tuning=PEFT + Continuous Pretraining2025.06 | 62 | — | |
| Qwen3-14BParameters=14B2025.10 | 61.48 | — | |
| CyberPal-2.0-4BParameters=4B2025.10 | 59.02 | — | |
| Qwen3-4BParameters=4B2025.10 | 57.38 | — | |
| Qwen3-8BParameters=8B2025.10 | 56.19 | — | |
| Llama-Primus-Base (Cybersecurity specialized)Size [B]=8, Base Alignment=SFT + RLHF, Extra Tuning=Continuous Pretraining2025.06 | 50 | — | |
| Mistral-7B-v0.1Size [B]=7, Base Alignment=None (pretrain only), Extra Tuning=None2025.06 | 43.7 | — | |
| Qwen-7BSize [B]=7, Base Alignment=None (pretrain only), Extra Tuning=None2025.06 | 31.4 | — |