Specification Alignment on SPECBENCH Average over scenarios
93.73Safety ScoreGPT-5-chat
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-5-chatModel Category=Instruct, Maximum generation length=4,2002025.09 | 93.73 | 82.54 | 82.14 | |
| STAIR-Llama-3.1-8B-DPO-3Model Category=Reasoning, Maximum generation length=8,4002025.09 | 89.27 | 36.86 | 48.25 | |
| Gemini-2.5-flash-thinkingModel Category=Reasoning, Variant=Thinking, Maximum generation length=8,4002025.09 | 84.73 | 73.49 | 68.59 | |
| RealSafe-R1-8BModel Category=Reasoning, Maximum generation length=8,4002025.09 | 83.6 | 37.37 | 45.93 | |
| Gemini-2.5-flash-lite-thinkingModel Category=Reasoning, Variant=Thinking, Maximum generation length=8,4002025.09 | 80 | 73.59 | 65.02 | |
| Gemini-2.5-proModel Category=Reasoning, Maximum generation length=8,4002025.09 | 79.6 | 73.89 | 65.03 | |
| Llama-3.2-1B-InstructModel Category=Instruct, Maximum generation length=4,2002025.09 | 79.33 | 27.3 | 37.61 | |
| GPT-4.1Model Category=Instruct, Maximum generation length=4,2002025.09 | 78.87 | 82.69 | 69.2 | |
| DeepSeek-R1Model Category=Reasoning, Maximum generation length=8,4002025.09 | 78.6 | 78.94 | 67.12 | |
| Qwen3-32B-thinkingModel Category=Reasoning, Variant=Thinking, Maximum generation length=8,4002025.09 | 71.8 | 75.98 | 60.12 | |
| GPT-4.1-miniModel Category=Instruct, Maximum generation length=4,2002025.09 | 71.4 | 72.38 | 57.24 | |
| Gemini-2.5-flashModel Category=Instruct, Maximum generation length=4,2002025.09 | 71.27 | 71.45 | 55.85 | |
| DeepSeek-V3Model Category=Instruct, Maximum generation length=4,2002025.09 | 70.13 | 72.99 | 57.65 | |
| Qwen3-14B-thinkingModel Category=Reasoning, Variant=Thinking, Maximum generation length=8,4002025.09 | 70 | 73.76 | 57.32 | |
| Qwen3-32BModel Category=Instruct, Maximum generation length=4,2002025.09 | 68.67 | 67.73 | 52.47 | |
| DeepSeek-R1-Distill-Llama-70BModel Category=Reasoning, Maximum generation length=8,4002025.09 | 68.53 | 55.32 | 47.8 | |
| Gemini-2.5-flash-liteModel Category=Instruct, Maximum generation length=4,2002025.09 | 68.33 | 63.8 | 50.15 | |
| Llama-3.3-70B-InstructModel Category=Instruct, Maximum generation length=4,2002025.09 | 68.07 | 69.38 | 53.8 | |
| Qwen3-8B-thinkingModel Category=Reasoning, Variant=Thinking, Maximum generation length=8,4002025.09 | 66.73 | 71.76 | 53.83 | |
| Qwen3-14BModel Category=Instruct, Maximum generation length=4,2002025.09 | 64.27 | 70.58 | 51.03 | |
| Llama-3.2-3B-InstructModel Category=Instruct, Maximum generation length=4,2002025.09 | 63.67 | 47.65 | 40.49 | |
| Qwen3-4B-thinkingModel Category=Reasoning, Variant=Thinking, Maximum generation length=8,4002025.09 | 63.47 | 68.04 | 50.26 | |
| Qwen3-8BModel Category=Instruct, Maximum generation length=4,2002025.09 | 62.53 | 66.88 | 48.58 | |
| Qwen3-4BModel Category=Instruct, Maximum generation length=4,2002025.09 | 59.13 | 59.54 | 43.33 | |
| DeepSeek-R1-Distill-Qwen-32BModel Category=Reasoning, Maximum generation length=8,4002025.09 | 57.8 | 52.82 | 39.53 | |
| Llama-3.1-8B-InstructModel Category=Instruct, Maximum generation length=4,2002025.09 | 56.87 | 65.99 | 44.54 | |
| Qwen3-0.6BModel Category=Instruct, Maximum generation length=4,2002025.09 | 56.47 | 31.85 | 30.74 | |
| Qwen3-1.7BModel Category=Instruct, Maximum generation length=4,2002025.09 | 55.27 | 47.48 | 36.34 | |
| Mistral-Small-Instruct-2409Model Category=Instruct, Maximum generation length=4,2002025.09 | 53.67 | 59.32 | 40.42 | |
| DeepSeek-R1-Distill-Llama-8BModel Category=Reasoning, Maximum generation length=8,4002025.09 | 53.67 | 45.44 | 35.01 | |
| Qwen3-0.6B-thinkingModel Category=Reasoning, Variant=Thinking, Maximum generation length=8,4002025.09 | 52.8 | 41.15 | 33.59 | |
| Qwen3-1.7B-thinkingModel Category=Reasoning, Variant=Thinking, Maximum generation length=8,4002025.09 | 52.6 | 54.81 | 37.8 | |
| Mistral-7B-Instruct-v0.3Model Category=Instruct, Maximum generation length=4,2002025.09 | 51.27 | 51.21 | 36.52 |