# Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models

Source: [Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models](<https://www.sota2.com/research/paper/68fa71ab1e209131cea52711>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:28:17.403Z

Authors: Shuang Li, Jiangjie Chen, Siyu Yuan, Xinyi Wu, Hao Yang, Shimin Tao, Yanghua Xiao

Published: 2023-08-26T21:38:31.000Z

## Abstract

To translate well, machine translation (MT) systems and general-purposed language models (LMs) need a deep understanding of both source and target languages and cultures. Therefore, idioms, with their non-compositional nature, pose particular challenges for Transformer-based systems, as literal translations often miss the intended meaning. Traditional methods, which replace idioms using existing knowledge bases (KBs), often lack scale and context awareness. Addressing these challenges, our approach prioritizes context awareness and scalability, allowing for offline storage of idioms in a manageable KB size. This ensures efficient serving with smaller models and provides a more comprehensive understanding of idiomatic expressions. We introduce a multilingual idiom KB (IdiomKB) developed using large LMs to address this. This KB facilitates better translation by smaller models, such as BLOOMZ (7.1B), Alpaca (7B), and InstructGPT (6.7B), by retrieving idioms' figurative meanings. We present a novel, GPT-4-powered metric for human-aligned evaluation, demonstrating that IdiomKB considerably boosts model performance. Human evaluations further validate our KB's quality.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Machine Translation on Seen Dataset en-ta 1.0](<https://www.sota2.com/research/sota/machine-translation-on-seen-dataset-en-ta-1-0.md>): Machine Translation; Seen Dataset en-ta 1.0; GPT-4 Score: 1.65.

- [Idiomatic Translation on en-hi](<https://www.sota2.com/research/sota/idiomatic-translation-on-en-hi.md>): Idiomatic Translation; en-hi; Human Score (1-5): 2.65.

- [Idiomatic Translation on en-bn](<https://www.sota2.com/research/sota/idiomatic-translation-on-en-bn.md>): Idiomatic Translation; en-bn; Human Score: 1.82.

- [Idiomatic Translation on en-tl](<https://www.sota2.com/research/sota/idiomatic-translation-on-en-tl.md>): Idiomatic Translation; en-tl; Human Score: 1.88.

- [Machine Translation on Seen Dataset en-bn 1.0](<https://www.sota2.com/research/sota/machine-translation-on-seen-dataset-en-bn-1-0.md>): Machine Translation; Seen Dataset en-bn 1.0; GPT-4 Score: 1.64.

- [Machine Translation on Seen Dataset en-te 1.0](<https://www.sota2.com/research/sota/machine-translation-on-seen-dataset-en-te-1-0.md>): Machine Translation; Seen Dataset en-te 1.0; GPT-4 Score: 1.5.

- [Machine Translation on Seen Dataset en-hi 1.0](<https://www.sota2.com/research/sota/machine-translation-on-seen-dataset-en-hi-1-0.md>): Machine Translation; Seen Dataset en-hi 1.0; GPT-4 Score: 1.875.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2308.13961>)

[PDF](<https://arxiv.org/pdf/2308.13961.pdf>)