# UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages

Source: [UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages](<https://www.sota2.com/research/paper/68fa748d1e209131cea626ca>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:53:29.253Z

Authors: Trinh Pham, Khoi M. Le, Luu Anh Tuan

Published: 2024-06-14T04:55:30.000Z

## Abstract

In this paper, we introduce UniBridge (Cross-Lingual Transfer Learning with Optimized Embeddings and Vocabulary), a comprehensive approach developed to improve the effectiveness of Cross-Lingual Transfer Learning, particularly in languages with limited resources. Our approach tackles two essential elements of a language model: the initialization of embeddings and the optimal vocabulary size. Specifically, we propose a novel embedding initialization method that leverages both lexical and semantic alignment for a language. In addition, we present a method for systematically searching for the optimal vocabulary size, ensuring a balance between model complexity and linguistic coverage. Our experiments across multilingual datasets show that our approach greatly improves the F1-Score in several languages. UniBridge is a robust and adaptable solution for cross-lingual systems in various languages, highlighting the significance of initializing embeddings and choosing the right vocabulary size in cross-lingual environments.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [POS Tagging on Universal Dependencies (UD)](<https://www.sota2.com/research/sota/pos-tagging-on-universal-dependencies-ud.md>): POS Tagging; Universal Dependencies (UD); AMH Accuracy: 40.88.

- [Natural Language Inference on AmericasNLI (test)](<https://www.sota2.com/research/sota/natural-language-inference-on-americasnli-test.md>): Natural Language Inference; AmericasNLI (test); Accuracy (aym): 52.13.

- [Named Entity Recognition on WikiAnn (test)](<https://www.sota2.com/research/sota/named-entity-recognition-on-wikiann-test.md>): Named Entity Recognition; WikiAnn (test); Average Accuracy: 61.81.

- [Part-Of-Speech Tagging on Universal Dependencies (UD) (test)](<https://www.sota2.com/research/sota/part-of-speech-tagging-on-universal-dependencies-ud-test.md>): Part-of-Speech Tagging; Universal Dependencies (UD) (test); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2406.09717>)

[PDF](<https://arxiv.org/pdf/2406.09717.pdf>)

[Implementation](<https://github.com/TokisakiKurumi2001/UniBridge>)