Cybersecurity Multiple Choice Question Answering on RedSage-MCQ (test)
88.68Macro AccuracyGPT-5
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GPT-5Input Template=Chat Template, Model Category=Larger Instruct & Proprietary Model, Shot-setting=0-shot2026.01 | 88.68 | 88.74 | 86.54 | 91.43 | 90.8 | 83.14 | |
| RedSage-8B-InsInput Template=Chat Template, Model Category=Instruct Model, Shot-setting=0-shot2026.01 | 85.73 | 84.2 | 84.98 | 89.06 | 86.8 | 80.3 | |
| Qwen3-32BInput Template=Chat Template, Model Category=Larger Instruct & Proprietary Model, Shot-setting=0-shot2026.01 | 85.4 | 84.08 | 82.32 | 89 | 87.6 | 80.4 | |
| RedSage-8B-SeedInput Template=Text Completion, Model Category=Base Model, Shot-setting=0-shot2026.01 | 85.21 | 83.64 | 84.56 | 88.82 | 85.5 | 79.9 | |
| RedSage-8B-BaseInput Template=Text Completion, Model Category=Base Model, Shot-setting=0-shot2026.01 | 85.05 | 83.12 | 84.94 | 88.72 | 85.44 | 79.36 | |
| RedSage-8B-CFWInput Template=Text Completion, Model Category=Base Model, Shot-setting=0-shot2026.01 | 84.86 | 83.62 | 83.3 | 88.81 | 85.3 | 79.32 | |
| RedSage-8B-DPOInput Template=Chat Template, Model Category=Instruct Model, Shot-setting=0-shot2026.01 | 84.83 | 82.48 | 83.8 | 88.54 | 86.3 | 79.3 | |
| Qwen3-8B-BaseInput Template=Text Completion, Model Category=Base Model, Shot-setting=0-shot2026.01 | 84.24 | 83.08 | 81.94 | 88.23 | 85.08 | 78.86 | |
| Qwen3-8BInput Template=Chat Template, Model Category=Instruct Model, Shot-setting=0-shot2026.01 | 81.85 | 80.46 | 78.82 | 86.16 | 83.92 | 75.56 | |
| DeepHat-V1-7BInput Template=Chat Template, Model Category=Instruct Model, Shot-setting=0-shot2026.01 | 80.18 | 77.26 | 76.9 | 85.07 | 81.94 | 74.82 | |
| Foundation-Sec-8BInput Template=Text Completion, Model Category=Base Model, Shot-setting=0-shot2026.01 | 78.51 | 76.82 | 79.1 | 83.68 | 76.64 | 71.14 | |
| Llama-3.1-8BInput Template=Text Completion, Model Category=Base Model, Shot-setting=0-shot2026.01 | 78.02 | 77.42 | 75.26 | 82.78 | 77.78 | 72.12 | |
| Llama-3.1-8B-InstructInput Template=Chat Template, Model Category=Instruct Model, Shot-setting=0-shot2026.01 | 77.05 | 76.06 | 73.3 | 80.9 | 78.72 | 72.4 | |
| Llama-Primus-BaseInput Template=Chat Template, Model Category=Instruct Model, Shot-setting=0-shot2026.01 | 77.02 | 76.78 | 74.1 | 80.87 | 76.78 | 72.72 | |
| Foundation-Sec-8B-InstructInput Template=Chat Template, Model Category=Instruct Model, Shot-setting=0-shot2026.01 | 76.12 | 74.5 | 77.1 | 80.91 | 74.98 | 68.3 | |
| Llama-Primus-MergedInput Template=Chat Template, Model Category=Instruct Model, Shot-setting=0-shot2026.01 | 74.81 | 74.34 | 72.34 | 79.31 | 74.74 | 68.82 | |
| Lily-Cybersecurity-7B-v0.2Input Template=Chat Template, Model Category=Instruct Model, Shot-setting=0-shot2026.01 | 71.19 | 68.78 | 67.44 | 76.61 | 71.44 | 66.26 |