Metric alignment with human overall quality judgements on Synthetics BSL
0.83Pearson rBT2 Fidelity
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BT2 FidelityCategory=Per-component2026.06 | 0.83 | 0.94 | |
| SignCLIP - BSL sentenceMetric configuration=pose-to-pose (p2p), Requirement=Reference-based (source sequence)2026.06 | 0.82 | 0.7 | |
| SignCLIP - Multilingual segmentationMetric configuration=pose-to-pose (p2p), Requirement=Reference-based (source sequence)2026.06 | 0.75 | 0.6 | |
| SignCLIP - BSL segmentationMetric configuration=pose-to-pose (p2p), Requirement=Reference-based (source sequence)2026.06 | 0.63 | 0.6 | |
| SignCLIP - Multilingual sentenceMetric configuration=pose-to-pose (p2p), Requirement=Reference-based (source sequence)2026.06 | 0.55 | 0.6 | |
| SVAERequirement=Reference-based (source sequence)2026.06 | 0.54 | 0.31 | |
| BT2 OverallRequirement=Reference-free (source text)2026.06 | 0.52 | 0.66 | |
| BT2 FluencyCategory=Per-component2026.06 | 0.4 | 0.54 | |
| BT2 PhonologyCategory=Per-component2026.06 | 0.35 | -0.03 | |
| SignCLIP - BSL sentenceMetric configuration=pose-to-text (p2t), Requirement=Reference-free (source text)2026.06 | 0.17 | 0.4 | |
| SiBLEURequirement=Reference-based (source sequence)2026.06 | -0.05 | 0.31 | |
| ROUGERequirement=Reference-free (source text)2026.06 | -0.08 | -0.1 | |
| BLEU-4Requirement=Reference-free (source text)2026.06 | -0.15 | -0.1 | |
| SignCLIP - BSL segmentationMetric configuration=pose-to-text (p2t), Requirement=Reference-free (source text)2026.06 | -0.2 | -0.1 | |
| DTW-MPJPERequirement=Reference-based (source sequence)2026.06 | -0.33 | -0.37 | |
| BLEURTRequirement=Reference-free (source text)2026.06 | -0.39 | -0.09 | |
| SignCLIP - Multilingual sentenceMetric configuration=pose-to-text (p2t), Requirement=Reference-free (source text)2026.06 | -0.41 | -0.3 | |
| BT2 GrammarCategory=Per-component2026.06 | -0.45 | -0.26 | |
| SignCLIP - Multilingual segmentationMetric configuration=pose-to-text (p2t), Requirement=Reference-free (source text)2026.06 | -0.5 | -0.3 | |
| ChrFRequirement=Reference-free (source text)2026.06 | -0.54 | -0.37 |