# Refusal Prediction on Locked GLM n=4,200, 20 item clusters (test)

Source: [Refusal Prediction on Locked GLM n=4,200, 20 item clusters (test)](<https://www.sota2.com/research/sota/refusal-prediction-on-locked-glm-n-4-200-20-item-clusters-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-08T11:01:57.725Z

Evaluation of Refusal Prediction performance using the specific test split of the Locked GLM dataset (4,200 samples, 20 item clusters), reporting statistical measures such as beta coefficients, standard error (SE), z-scores, and p-values.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Refusal Prediction |
| Dataset (including any stated split) | Locked GLM n=4,200, 20 item clusters (test) |
| Metric | Beta Coefficient |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 4 of 4 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Beta Coefficient](<https://www.sota2.com/research/sota/refusal-prediction-on-locked-glm-n-4-200-20-item-clusters-test.md?metric=Beta%20Coefficient>): Higher is better; unit: SCALAR.

- [SE](<https://www.sota2.com/research/sota/refusal-prediction-on-locked-glm-n-4-200-20-item-clusters-test.md?metric=SE>): Lower is better; unit: SCALAR.

- [Z-score](<https://www.sota2.com/research/sota/refusal-prediction-on-locked-glm-n-4-200-20-item-clusters-test.md?metric=Z-score>): Higher is better; unit: SCALAR.

- [P-value](<https://www.sota2.com/research/sota/refusal-prediction-on-locked-glm-n-4-200-20-item-clusters-test.md?metric=P-value>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Beta Coefficient | Result date | Report paper |
| --- | --- | --- | --- |
| Llama-3.1-70B | -0.475 | Not recorded | 69fd6e555497b3b670c89695 |
| Phi-3.5-mini | -0.541 | Not recorded | 69fd6e555497b3b670c89695 |
| Mistral-Small-3.2 | -0.791 | Not recorded | 69fd6e555497b3b670c89695 |
| Llama-3.1-8B | -1.154 | Not recorded | 69fd6e555497b3b670c89695 |

### Llama-3.1-70B

Beta Coefficient: -0.475. Result ID: 69fdc29947f34ce924e18ab9.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6e555497b3b670c89695.md>)

Recorded evaluation configuration: {"Reference Model":"OLMo-3-Instruct","Frame Reference":"deployment","Interaction term":"model × evaluation"}. This record does not assert independent reproduction.

### Phi-3.5-mini

Beta Coefficient: -0.541. Result ID: 69fdc29947f34ce924e18ab7.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6e555497b3b670c89695.md>)

Recorded evaluation configuration: {"Reference Model":"OLMo-3-Instruct","Frame Reference":"deployment","Interaction term":"model × evaluation"}. This record does not assert independent reproduction.

### Mistral-Small-3.2

Beta Coefficient: -0.791. Result ID: 69fdc29947f34ce924e18ab6.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6e555497b3b670c89695.md>)

Recorded evaluation configuration: {"Reference Model":"OLMo-3-Instruct","Frame Reference":"deployment","Interaction term":"model × evaluation"}. This record does not assert independent reproduction.

### Llama-3.1-8B

Beta Coefficient: -1.154. Result ID: 69fdc29947f34ce924e18ab8.

[Reporting paper](<https://www.sota2.com/research/paper/69fd6e555497b3b670c89695.md>)

Recorded evaluation configuration: {"Reference Model":"OLMo-3-Instruct","Frame Reference":"deployment","Interaction term":"model × evaluation"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.