Cybersecurity Knowledge Evaluation on CTI-MCQ
71.8AccuracyLlama-3.3-70B-Ins-DAP
Evaluation results
| Method | Links | |
|---|---|---|
| Llama-3.3-70B-Ins-DAPSize [B]=70, Base Alignment=SFT + RLHF, Extra Tuning=DAP2025.06 | 71.8 | |
| DSR1D-Qwen-14B-DAPSize [B]=14, Base Alignment=KD + SFT, Extra Tuning=DAP2025.06 | 68.5 | |
| Foundation-Sec-8B (Cybersecurity specialized)Size [B]=8, Base Alignment=SFT + RLHF, Extra Tuning=Continuous Pretraining2025.06 | 67.6 | |
| Llama-Primus-Base (Cybersecurity specialized)Size [B]=8, Base Alignment=SFT + RLHF, Extra Tuning=Continuous Pretraining2025.06 | 66.7 | |
| Llama-3-70B-InstructSize [B]=70, Base Alignment=SFT + (multi-round) DPO, Extra Tuning=None2025.06 | 65.7 | |
| Llama-3.1-8B-DAPSize [B]=8, Base Alignment=None (pretrain only), Extra Tuning=DAP2025.06 | 64.2 | |
| Llama-3-8B-InstructSize [B]=8, Base Alignment=SFT + RLHF, Extra Tuning=None2025.06 | 61.3 | |
| Llama-3-8B-BaseSize [B]=8, Base Alignment=None (pretrain only), Extra Tuning=PEFT + Continuous Pretraining2025.06 | 49.9 |