Response Quality Evaluation on Held-out n=300 (test)
0.747Pearson Correlation CoefficientDeBERTa Judge
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeBERTa JudgeParams=184M, Latency=14.8 ms2026.04 | 0.747 | 0.663 | 0.733 | |
| MiniLM JudgeParams=22M, Latency=13.3 ms2026.04 | 0.676 | 0.567 | 0.685 | |
| STS-stsbParams=66M, Latency=∼0.8 ms2026.04 | 0.647 | 0.578 | 0.698 | |
| STS-paraphraseParams=22M, Latency=∼0.7 ms2026.04 | 0.629 | 0.554 | 0.683 | |
| STS-MiniLMParams=22M, Latency=∼0.7 ms2026.04 | 0.629 | 0.558 | 0.7 | |
| TextCNN JudgeParams=10M, Latency=1.0 ms2026.04 | 0.472 | 0.346 | 0.452 | |
| CE-MiniLMParams=22M, Latency=∼0.5 ms2026.04 | 0.331 | 0.238 | 0.357 | |
| CE-DeBERTaParams=184M, Latency=∼7.3 ms2026.04 | -0.363 | 0.48 | -0.25 |