# Sakura at BEA 2026 Shared Task 1: What Makes Vocabulary Difficult?

Source: [Sakura at BEA 2026 Shared Task 1: What Makes Vocabulary Difficult?](<https://www.sota2.com/research/paper/6a06a8a3dbc836b87d82a120>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-15T05:00:09.989Z

Authors: Adam Nohejl, Xuanxin Wu, Yusuke Ide, Maria Angelica Riera Machin, Yi-Ning Chang, Hitomi Yanaka

Published: 2026-05-14T01:57:35.000Z

## Abstract

We describe two types of models for vocabulary difficulty prediction: a high-accuracy black-box model, which achieved the top shared task result in the open track, and an explainable model, which outperforms a fine-tuned encoder baseline. As the black-box model, we fine-tuned an LLM using a soft-target loss function for effective application to the rating task, achieving r \> 0.91. The explainable model provides insights into what impacts the difficulty of each item while maintaining a strong correlation (r \> 0.77). We further analyze the results, demonstrating that the difficulty of items in the British Council's Knowledge-based Vocabulary Lists (KVL) is often affected by spelling difficulty or the construction of the test items, in addition to the genuine production difficulty of the words. We make our code available online at https://github.com/ynklab/vocabulary-difficulty .

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Vocabulary Difficulty Prediction on KVL data (test)](<https://www.sota2.com/research/sota/vocabulary-difficulty-prediction-on-kvl-data-test.md>): Vocabulary difficulty prediction; KVL data (test); PCC (Chinese): 0.928.

- [Vocabulary Difficulty Prediction on KVL BEA Shared Task Open Track 2026 (test)](<https://www.sota2.com/research/sota/vocabulary-difficulty-prediction-on-kvl-bea-shared-task-open-track-2026-test.md>): Vocabulary difficulty prediction; KVL BEA Shared Task Open Track 2026 (test); Chinese RMSE: 0.63.

- [Lexical Complexity Prediction on BEA Shared Task Closed Track 2026 (test)](<https://www.sota2.com/research/sota/lexical-complexity-prediction-on-bea-shared-task-closed-track-2026-test.md>): Lexical Complexity Prediction; BEA Shared Task Closed Track 2026 (test); Chinese Score: 1.078.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.14257>)

[PDF](<https://arxiv.org/pdf/2605.14257.pdf>)