# A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change

Source: [A Systematic Comparison of Contextualized Word Embeddings for Lexical Semantic Change](<https://www.sota2.com/research/paper/68fa75741e209131cea6804a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:34:52.943Z

Authors: Francesco Periti, Nina Tahmasebi

Published: 2024-02-19T10:04:59.000Z

## Abstract

Contextualized embeddings are the preferred tool for modeling Lexical Semantic Change (LSC). Current evaluations typically focus on a specific task known as Graded Change Detection (GCD). However, performance comparison across work are often misleading due to their reliance on diverse settings. In this paper, we evaluate state-of-the-art models and approaches for GCD under equal conditions. We further break the LSC problem into Word-in-Context (WiC) and Word Sense Induction (WSI) tasks, and compare models across these different levels. Our evaluation is performed across different languages on eight available benchmarks for LSC, and shows that (i) APD outperforms other approaches for GCD; (ii) XL-LEXEME outperforms other contextualized models for WiC, WSI, and GCD, while being comparable to GPT-4; (iii) there is a clear need for improving the modeling of word meanings, as well as focus on how, when, and why these meanings change, rather than solely focusing on the extent of semantic change.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Measuring magnitude of word-level semantic change on DWUG English](<https://www.sota2.com/research/sota/measuring-magnitude-of-word-level-semantic-change-on-dwug-english.md>): Measuring magnitude of word-level semantic change; DWUG English; Spearman Correlation: 0.71.

- [Word-level semantic change magnitude measurement on DWUG Spanish 2022](<https://www.sota2.com/research/sota/word-level-semantic-change-magnitude-measurement-on-dwug-spanish-2022.md>): Word-level semantic change magnitude measurement; DWUG Spanish 2022; Spearman Correlation: 0.61.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2402.12011>)

[PDF](<https://arxiv.org/pdf/2402.12011.pdf>)