# LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution

Source: [LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution](<https://www.sota2.com/research/paper/68fa6c301e209131cea32b23>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-24T17:01:53.999Z

Authors: George Michalopoulos, Ian McKillop, Alexander Wong, Helen Chen

Published: 2021-07-11T21:25:56.000Z

## Abstract

Lexical substitution is the task of generating meaningful substitutes for a word in a given textual context. Contextual word embedding models have achieved state-of-the-art results in the lexical substitution task by relying on contextual information extracted from the replaced word within the sentence. However, such models do not take into account structured knowledge that exists in external lexical databases.   We introduce LexSubCon, an end-to-end lexical substitution framework based on contextual embedding models that can identify highly accurate substitute candidates. This is achieved by combining contextual information with knowledge from structured lexical resources. Our approach involves: (i) introducing a novel mix-up embedding strategy in the creation of the input embedding of the target word through linearly interpolating the pair of the target input embedding and the average embedding of its probable synonyms; (ii) considering the similarity of the sentence-definition embeddings of the target word and its proposed candidates; and, (iii) calculating the effect of each substitution in the semantics of the sentence through a fine-tuned sentence similarity model. Our experiments show that LexSubCon outperforms previous state-of-the-art methods on LS07 and CoInCo benchmark datasets that are widely used for lexical substitution tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Lexical Substitution on LS07 (test)](<https://www.sota2.com/research/sota/lexical-substitution-on-ls07-test.md>): Lexical Substitution; LS07 (test); Best Score: 21.1.

- [Lexical Substitution on CoInCo (test)](<https://www.sota2.com/research/sota/lexical-substitution-on-coinco-test.md>): Lexical Substitution; CoInCo (test); Best Score: 14.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2107.05132>)

[PDF](<https://arxiv.org/pdf/2107.05132.pdf>)

[Implementation](<https://github.com/gmichalo/LexSubCon>)