# Machine Translation Evaluation on TAC summary-level 2008-2011 (test) Source: [Machine Translation Evaluation on TAC summary-level 2008-2011 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:57:41.750Z Evaluation of machine translation quality metrics, specifically Pearson correlation scores (Pyramid and Responsiveness), applied to the TAC summary-level dataset covering the 2008-2011 test splits. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Machine Translation Evaluation | | Dataset (including any stated split) | TAC summary-level 2008-2011 (test) | | Metric | Pearson Correlation (Pyramid) | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pearson Correlation (Pyramid)](): Higher is better; unit: SCALAR. - [Pearson Correlation (Responsiveness)](): Higher is better; unit: SCALAR. ## Recorded results | Method | Pearson Correlation (Pyramid) | Result date | Report paper | | --- | --- | --- | --- | | FrugalScore | 67.3 | Not recorded | 68fa6b131e209131cea2bfd7 | | MoverScore | 66.5 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 66.2 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 66 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 65.9 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 65.5 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 64.9 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 64.8 | Not recorded | 68fa6b131e209131cea2bfd7 | | BERTScore | 64.7 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 64.7 | Not recorded | 68fa6b131e209131cea2bfd7 | | BERTScore | 64.5 | Not recorded | 68fa6b131e209131cea2bfd7 | | BERTScore | 64.2 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 64.1 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 63.9 | Not recorded | 68fa6b131e209131cea2bfd7 | | BERTScore | 62.7 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 61.7 | Not recorded | 68fa6b131e209131cea2bfd7 | | BERTScore | 61.6 | Not recorded | 68fa6b131e209131cea2bfd7 | | FrugalScore | 60 | Not recorded | 68fa6b131e209131cea2bfd7 | | BERTScore | 55.4 | Not recorded | 68fa6b131e209131cea2bfd7 | ### FrugalScore Pearson Correlation (Pyramid): 67.3. Result ID: 698e5310ffc3901de09ca1c2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Tiny","Teacher":"MoverScore (BERT-Base)","Params":"4.4M","Runtime":"1m 28s"}. This record does not assert independent reproduction. ### MoverScore Pearson Correlation (Pyramid): 66.5. Result ID: 698e5310ffc3901de09ca1b8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BERT-Base","Params":"110M","Runtime":"301m 29s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 66.2. Result ID: 698e5310ffc3901de09ca1c4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Medium","Teacher":"MoverScore (BERT-Base)","Params":"41.7M","Runtime":"3m 41s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 66. Result ID: 698e5310ffc3901de09ca1c0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Small","Teacher":"BERTScore (DeBERTa-XLarge)","Params":"29.1M","Runtime":"2m 29s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 65.9. Result ID: 698e5310ffc3901de09ca1c3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Small","Teacher":"MoverScore (BERT-Base)","Params":"29.1M","Runtime":"2m 29s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 65.5. Result ID: 698e5310ffc3901de09ca1c1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Medium","Teacher":"BERTScore (DeBERTa-XLarge)","Params":"41.7M","Runtime":"3m 41s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 64.9. Result ID: 698e5310ffc3901de09ca1b9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Tiny","Teacher":"BERTScore (BERT-Base)","Params":"4.4M","Runtime":"1m 28s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 64.8. Result ID: 698e5310ffc3901de09ca1bb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Medium","Teacher":"BERTScore (BERT-Base)","Params":"41.7M","Runtime":"3m 41s"}. This record does not assert independent reproduction. ### BERTScore Pearson Correlation (Pyramid): 64.7. Result ID: 698e5310ffc3901de09ca1b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BERT-Base","Params":"110M","Runtime":"3m 28s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 64.7. Result ID: 698e5310ffc3901de09ca1ba. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Small","Teacher":"BERTScore (BERT-Base)","Params":"29.1M","Runtime":"2m 29s"}. This record does not assert independent reproduction. ### BERTScore Pearson Correlation (Pyramid): 64.5. Result ID: 698e5310ffc3901de09ca1b7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeBERTa-XLarge","Params":"900M","Runtime":"6m 20s"}. This record does not assert independent reproduction. ### BERTScore Pearson Correlation (Pyramid): 64.2. Result ID: 698e5310ffc3901de09ca1b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"RoBERTa-Large","Params":"355M","Runtime":"5m 17s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 64.1. Result ID: 698e5310ffc3901de09ca1bd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Small","Teacher":"BERTScore (RoBERTa-Large)","Params":"29.1M","Runtime":"2m 29s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 63.9. Result ID: 698e5310ffc3901de09ca1be. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Medium","Teacher":"BERTScore (RoBERTa-Large)","Params":"41.7M","Runtime":"3m 41s"}. This record does not assert independent reproduction. ### BERTScore Pearson Correlation (Pyramid): 62.7. Result ID: 698e5310ffc3901de09ca1b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BERT-Medium","Params":"41.7M","Runtime":"2m 28s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 61.7. Result ID: 698e5310ffc3901de09ca1bf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Tiny","Teacher":"BERTScore (DeBERTa-XLarge)","Params":"4.4M","Runtime":"1m 28s"}. This record does not assert independent reproduction. ### BERTScore Pearson Correlation (Pyramid): 61.6. Result ID: 698e5310ffc3901de09ca1b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BERT-Small","Params":"29.1M","Runtime":"2m 20s"}. This record does not assert independent reproduction. ### FrugalScore Pearson Correlation (Pyramid): 60. Result ID: 698e5310ffc3901de09ca1bc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"BERT-Tiny","Teacher":"BERTScore (RoBERTa-Large)","Params":"4.4M","Runtime":"1m 28s"}. This record does not assert independent reproduction. ### BERTScore Pearson Correlation (Pyramid): 55.4. Result ID: 698e5310ffc3901de09ca1b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"BERT-Tiny","Params":"4.4M","Runtime":"1m 27s"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.