Specification Alignment on Specification Alignment Evaluation Set (full (1500))
76.4Safety ScoreAlign3
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Align3Base Model=Qwen3-14B-thinking, Avg. Tokens=18322025.09 | 76.4 | 74.84 | 62.92 | |
| MoreThinkBase Model=Qwen3-14B-thinking, Avg. Tokens=18372025.09 | 70.07 | 73.45 | 57.3 | |
| VanillaBase Model=Qwen3-14B-thinking, Avg. Tokens=15502025.09 | 70 | 73.76 | 57.32 | |
| TPOBase Model=Qwen3-14B, Avg. Tokens=215832025.09 | 68.53 | 78.28 | 58.76 | |
| Self-RefineBase Model=Qwen3-14B, Avg. Tokens=376262025.09 | 67.2 | 77.59 | 57.97 | |
| VanillaBase Model=Qwen3-14B, Avg. Tokens=9462025.09 | 64.27 | 70.58 | 51.03 | |
| ZeroThinkBase Model=Qwen3-14B-thinking, Avg. Tokens=9462025.09 | 64.27 | 70.58 | 51.03 | |
| Best-of-NBase Model=Qwen3-14B, Avg. Tokens=142312025.09 | 64.2 | 75.29 | 53.21 | |
| Align3Base Model=DeepSeek-R1-Distill-Llama-8B, Avg. Tokens=13692025.09 | 58.67 | 56.97 | 42.75 | |
| TPOBase Model=Llama-3.1-8B-Instruct, Avg. Tokens=169172025.09 | 57.27 | 72.06 | 48.03 | |
| Best-of-NBase Model=Llama-3.1-8B-Instruct, Avg. Tokens=122052025.09 | 57.2 | 71.92 | 47.71 | |
| VanillaBase Model=Llama-3.1-8B-Instruct, Avg. Tokens=7982025.09 | 56.87 | 65.99 | 44.54 | |
| MoreThinkBase Model=DeepSeek-R1-Distill-Llama-8B, Avg. Tokens=16112025.09 | 55.67 | 47.15 | 36.95 | |
| ZeroThinkBase Model=DeepSeek-R1-Distill-Llama-8B, Avg. Tokens=6912025.09 | 55.53 | 45.65 | 35.99 | |
| VanillaBase Model=DeepSeek-R1-Distill-Llama-8B, Avg. Tokens=13122025.09 | 53.67 | 45.44 | 35.01 | |
| Self-RefineBase Model=Llama-3.1-8B-Instruct, Avg. Tokens=341992025.09 | 52.8 | 43.45 | 35.16 |