# Speech Translation on Callhome En-Es (test) Source: [Speech Translation on Callhome En-Es (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:36:53.827Z Evaluation of Speech Translation performance using the English-to-Spanish (En-Es) test split of the Callhome corpus. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Speech Translation | | Dataset (including any stated split) | Callhome En-Es (test) | | Metric | BLEU | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [BLEU](): Higher is better; unit: PERCENT. - [Speed-up Ratio](): Higher is better; unit: SCALAR. ## Recorded results | Method | BLEU | Result date | Report paper | | --- | --- | --- | --- | | Transformer (MT Baseline) | 31.58 | Not recorded | 68fa6f781e209131cea45c96 | | CTC-NAST | 23.3 | Not recorded | 68fa6f781e209131cea45c96 | | CTC-Aug ST + Seq-KD | 22.92 | Not recorded | 68fa6f781e209131cea45c96 | | CTC-Aug ST | 21.33 | Not recorded | 68fa6f781e209131cea45c96 | | Conformer + Seq-KD | 21 | Not recorded | 68fa6f781e209131cea45c96 | | Orthros - CTC | 20.8 | Not recorded | 68fa6f781e209131cea45c96 | | Orthros - CMLM | 20.4 | Not recorded | 68fa6f781e209131cea45c96 | | Transformer + Seq KD | 19.81 | Not recorded | 68fa6f781e209131cea45c96 | | Transformer + Seq-KD | 19.2 | Not recorded | 68fa6f781e209131cea45c96 | | Self-conditioned CTC | 19.2 | Not recorded | 68fa6f781e209131cea45c96 | | Intermediate CTC | 19 | Not recorded | 68fa6f781e209131cea45c96 | | Conformer - CTC | 18.7 | Not recorded | 68fa6f781e209131cea45c96 | | Mask - CTC | 18.3 | Not recorded | 68fa6f781e209131cea45c96 | | Transformer + MTL + ASR init. | 17.45 | Not recorded | 68fa6f781e209131cea45c96 | | CTC | 15.91 | Not recorded | 68fa6f781e209131cea45c96 | | CTC + MTL | 14.19 | Not recorded | 68fa6f781e209131cea45c96 | | Transformer - CTC | 13.52 | Not recorded | 68fa6f781e209131cea45c96 | ### Transformer (MT Baseline) BLEU: 31.58. Result ID: 6990d6173a6c31484903a8b2. [Reporting paper]() Recorded evaluation configuration: {"Paradigm":"MT"}. This record does not assert independent reproduction. ### CTC-NAST BLEU: 23.3. Result ID: 6990d6173a6c31484903a8c2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Paradigm":"NAR"}. This record does not assert independent reproduction. ### CTC-Aug ST + Seq-KD BLEU: 22.92. Result ID: 6990d6173a6c31484903a8b8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Paradigm":"AR"}. This record does not assert independent reproduction. ### CTC-Aug ST BLEU: 21.33. Result ID: 6990d6173a6c31484903a8b7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Paradigm":"AR"}. This record does not assert independent reproduction. ### Conformer + Seq-KD BLEU: 21. Result ID: 6990d6173a6c31484903a8b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"AR","Backbone":"Conformer","Reference":"Inaguma et al., 2021a"}. This record does not assert independent reproduction. ### Orthros - CTC BLEU: 20.8. Result ID: 6990d6173a6c31484903a8bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"NAR","Reference":"Inaguma et al., 2021a"}. This record does not assert independent reproduction. ### Orthros - CMLM BLEU: 20.4. Result ID: 6990d6173a6c31484903a8bc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"NAR","Reference":"Inaguma et al., 2021a"}. This record does not assert independent reproduction. ### Transformer + Seq KD BLEU: 19.81. Result ID: 6990d6173a6c31484903a8b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"AR","Backbone":"Transformer","Reference":"Inaguma et al., 2021b"}. This record does not assert independent reproduction. ### Transformer + Seq-KD BLEU: 19.2. Result ID: 6990d6173a6c31484903a8b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"AR","Backbone":"Transformer","Reference":"Inaguma et al., 2021a"}. This record does not assert independent reproduction. ### Self-conditioned CTC BLEU: 19.2. Result ID: 6990d6173a6c31484903a8c1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"NAR","Reference":"Higuchi et al., 2021a"}. This record does not assert independent reproduction. ### Intermediate CTC BLEU: 19. Result ID: 6990d6173a6c31484903a8c0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"NAR","Reference":"Higuchi et al., 2021a"}. This record does not assert independent reproduction. ### Conformer - CTC BLEU: 18.7. Result ID: 6990d6173a6c31484903a8ba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"NAR","Backbone":"Conformer","Reference":"Inaguma et al., 2021a"}. This record does not assert independent reproduction. ### Mask - CTC BLEU: 18.3. Result ID: 6990d6173a6c31484903a8bf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"NAR","Reference":"Higuchi et al., 2021a"}. This record does not assert independent reproduction. ### Transformer + MTL + ASR init. BLEU: 17.45. Result ID: 6990d6173a6c31484903a8b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"AR","Backbone":"Transformer","Reference":"Chuang et al., 2021"}. This record does not assert independent reproduction. ### CTC BLEU: 15.91. Result ID: 6990d6173a6c31484903a8b9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"NAR","Reference":"Inaguma et al., 2021b"}. This record does not assert independent reproduction. ### CTC + MTL BLEU: 14.19. Result ID: 6990d6173a6c31484903a8be. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"NAR","Reference":"Chuang et al., 2021"}. This record does not assert independent reproduction. ### Transformer - CTC BLEU: 13.52. Result ID: 6990d6173a6c31484903a8bd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Paradigm":"NAR","Backbone":"Transformer","Reference":"Chuang et al., 2021"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.