# Response Quality Evaluation on Held-out n=300 (test)

Source: [Response Quality Evaluation on Held-out n=300 (test)](<https://www.sota2.com/research/sota/response-quality-evaluation-on-held-out-n-300-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-11T09:07:14.214Z

Evaluation of model response quality using Pearson and Spearman correlation coefficients on a held-out test set containing 300 samples.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Response Quality Evaluation |
| Dataset (including any stated split) | Held-out n=300 (test) |
| Metric | Pearson Correlation Coefficient |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Pearson Correlation Coefficient](<https://www.sota2.com/research/sota/response-quality-evaluation-on-held-out-n-300-test.md?metric=Pearson%20Correlation%20Coefficient>): Higher is better; unit: SCALAR.

- [Pearson 95% CI (Lower Bound)](<https://www.sota2.com/research/sota/response-quality-evaluation-on-held-out-n-300-test.md?metric=Pearson%2095%25%20CI%20(Lower%20Bound)>): Higher is better; unit: SCALAR.

- [Spearman Correlation Coefficient](<https://www.sota2.com/research/sota/response-quality-evaluation-on-held-out-n-300-test.md?metric=Spearman%20Correlation%20Coefficient>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Pearson Correlation Coefficient | Result date | Report paper |
| --- | --- | --- | --- |
| DeBERTa Judge | 0.747 | Not recorded | 6a2a40e2d33dfb23e783912b |
| MiniLM Judge | 0.676 | Not recorded | 6a2a40e2d33dfb23e783912b |
| STS-stsb | 0.647 | Not recorded | 6a2a40e2d33dfb23e783912b |
| STS-paraphrase | 0.629 | Not recorded | 6a2a40e2d33dfb23e783912b |
| STS-MiniLM | 0.629 | Not recorded | 6a2a40e2d33dfb23e783912b |
| TextCNN Judge | 0.472 | Not recorded | 6a2a40e2d33dfb23e783912b |
| CE-MiniLM | 0.331 | Not recorded | 6a2a40e2d33dfb23e783912b |
| CE-DeBERTa | -0.363 | Not recorded | 6a2a40e2d33dfb23e783912b |

### DeBERTa Judge

Pearson Correlation Coefficient: 0.747. Result ID: 6a2a7ac1fbc373592c27d282.

[Reporting paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

Recorded evaluation configuration: {"Params":"184M","Latency":"14.8 ms"}. This record does not assert independent reproduction.

### MiniLM Judge

Pearson Correlation Coefficient: 0.676. Result ID: 6a2a7ac1fbc373592c27d281.

[Reporting paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

Recorded evaluation configuration: {"Params":"22M","Latency":"13.3 ms"}. This record does not assert independent reproduction.

### STS-stsb

Pearson Correlation Coefficient: 0.647. Result ID: 6a2a7ac1fbc373592c27d284.

[Reporting paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a90e182d603e6de3e4091b.md>)

Recorded evaluation configuration: {"Params":"66M","Latency":"∼0.8 ms"}. This record does not assert independent reproduction.

### STS-paraphrase

Pearson Correlation Coefficient: 0.629. Result ID: 6a2a7ac1fbc373592c27d283.

[Reporting paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a90e182d603e6de3e4091b.md>)

Recorded evaluation configuration: {"Params":"22M","Latency":"∼0.7 ms"}. This record does not assert independent reproduction.

### STS-MiniLM

Pearson Correlation Coefficient: 0.629. Result ID: 6a2a7ac1fbc373592c27d285.

[Reporting paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a90e182d603e6de3e4091b.md>)

Recorded evaluation configuration: {"Params":"22M","Latency":"∼0.7 ms"}. This record does not assert independent reproduction.

### TextCNN Judge

Pearson Correlation Coefficient: 0.472. Result ID: 6a2a7ac1fbc373592c27d280.

[Reporting paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

Recorded evaluation configuration: {"Params":"10M","Latency":"1.0 ms"}. This record does not assert independent reproduction.

### CE-MiniLM

Pearson Correlation Coefficient: 0.331. Result ID: 6a2a7ac1fbc373592c27d286.

[Reporting paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a90e182d603e6de3e4091b.md>)

Recorded evaluation configuration: {"Params":"22M","Latency":"∼0.5 ms"}. This record does not assert independent reproduction.

### CE-DeBERTa

Pearson Correlation Coefficient: -0.363. Result ID: 6a2a7ac1fbc373592c27d287.

[Reporting paper](<https://www.sota2.com/research/paper/6a2a40e2d33dfb23e783912b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a90e182d603e6de3e4091b.md>)

Recorded evaluation configuration: {"Params":"184M","Latency":"∼7.3 ms"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.