Reasoning on GSM8K (pass@1)
90.58Pass@1SafeKey
Evaluation results
| Method | Links | |
|---|---|---|
| SafeKeyModel Size=7B, Sampling trajectories=82026.01 | 90.58 | |
| InitialModel Size=7B, Sampling trajectories=82026.01 | 90.32 | |
| STAR-1Model Size=7B, Sampling trajectories=82026.01 | 90.32 | |
| THINKSAFEModel Size=7B, Sampling trajectories=82026.01 | 90.1 | |
| SafeChainModel Size=7B, Sampling trajectories=82026.01 | 89.75 | |
| SafePathModel Size=7B, Sampling trajectories=82026.01 | 89.71 | |
| DirectRefusalModel Size=7B, Sampling trajectories=82026.01 | 88.27 | |
| InitialModel Size=8B, Sampling trajectories=82026.01 | 87.74 | |
| THINKSAFEModel Size=8B, Sampling trajectories=82026.01 | 87.55 | |
| SafePathModel Size=8B, Sampling trajectories=82026.01 | 87.45 | |
| SafeKeyModel Size=8B, Sampling trajectories=82026.01 | 87.41 | |
| STAR-1Model Size=8B, Sampling trajectories=82026.01 | 87.28 | |
| SafeChainModel Size=8B, Sampling trajectories=82026.01 | 86.06 | |
| DirectRefusalModel Size=8B, Sampling trajectories=82026.01 | 83.26 | |
| THINKSAFEModel Size=1.5B, Sampling trajectories=82026.01 | 82.58 | |
| InitialModel Size=1.5B, Sampling trajectories=82026.01 | 82.42 | |
| SafePathModel Size=1.5B, Sampling trajectories=82026.01 | 82.37 | |
| STAR-1Model Size=1.5B, Sampling trajectories=82026.01 | 81.38 | |
| SafeKeyModel Size=1.5B, Sampling trajectories=82026.01 | 81.25 | |
| DirectRefusalModel Size=1.5B, Sampling trajectories=82026.01 | 81.06 | |
| SafeChainModel Size=1.5B, Sampling trajectories=82026.01 | 80.47 |