Metric alignment with human understandability judgements on Human SRT BSL
0.96Pearson Correlation Coefficient (r)SiBLEU
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SiBLEURequirement=Reference-based (source sequence)2026.06 | 0.96 | 0.9 | |
| BT2 PhonologyCategory=Per-component2026.06 | 0.87 | 0.9 | |
| SVAERequirement=Reference-based (source sequence)2026.06 | 0.81 | 0.8 | |
| SignCLIP - BSL sentenceMetric configuration=pose-to-pose (p2p), Requirement=Reference-based (source sequence)2026.06 | 0.69 | 0.8 | |
| SignCLIP - BSL sentenceMetric configuration=pose-to-text (p2t), Requirement=Reference-free (source text)2026.06 | 0.56 | 0.6 | |
| BLEURTRequirement=Reference-free (source text)2026.06 | 0.15 | 0.35 | |
| DTW-MPJPERequirement=Reference-based (source sequence)2026.06 | 0.04 | 0 | |
| BLEU-4Requirement=Reference-free (source text)2026.06 | 0 | 0 | |
| ROUGERequirement=Reference-free (source text)2026.06 | 0 | 0 | |
| SignCLIP - Multilingual sentenceMetric configuration=pose-to-text (p2t), Requirement=Reference-free (source text)2026.06 | -0.07 | -0.1 | |
| SignCLIP - Multilingual sentenceMetric configuration=pose-to-pose (p2p), Requirement=Reference-based (source sequence)2026.06 | -0.1 | -0.4 | |
| SignCLIP - Multilingual segmentationMetric configuration=pose-to-text (p2t), Requirement=Reference-free (source text)2026.06 | -0.11 | -0.3 | |
| ChrFRequirement=Reference-free (source text)2026.06 | -0.15 | -0.35 | |
| SignCLIP - Multilingual segmentationMetric configuration=pose-to-pose (p2p), Requirement=Reference-based (source sequence)2026.06 | -0.3 | 0 | |
| BT2 OverallRequirement=Reference-free (source text)2026.06 | -0.43 | -0.3 | |
| BT2 GrammarCategory=Per-component2026.06 | -0.45 | -0.6 | |
| SignCLIP - BSL segmentationMetric configuration=pose-to-pose (p2p), Requirement=Reference-based (source sequence)2026.06 | -0.46 | -0.4 | |
| BT2 FluencyCategory=Per-component2026.06 | -0.46 | -0.4 | |
| SignCLIP - BSL segmentationMetric configuration=pose-to-text (p2t), Requirement=Reference-free (source text)2026.06 | -0.78 | -0.7 | |
| BT2 FidelityCategory=Per-component2026.06 | -0.81 | -0.9 |