# Model alignment using inter-modal bridges

Source: [Model alignment using inter-modal bridges](<https://www.sota2.com/research/paper/68fa789a1e209131cea793e1>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:42:14.951Z

Authors: Ali Gholamzadeh, Noor Sajid

Published: 2025-05-18T09:30:02.000Z

## Abstract

Foundation models have demonstrated remarkable performance across modalities such as language and vision. However, model reuse across distinct modalities (e.g., text and vision) remains limited due to the difficulty of aligning internal representations. Existing methods require extensive paired training data or are constrained to specific domains. We introduce a semi-supervised approach for model alignment via conditional flow matching. The conditional flow between latent spaces of different modalities (e.g., text-to-image or biological-to-artificial neuronal activity) can be learned in two settings: ($1$) solving a (balanced or unbalanced) optimal transport problem with an inter-space bridge cost, and ($2$) performing memory-efficient alignment using labelled exemplars. Despite being constrained by the original models' capacity, our method--under both settings--matches downstream task performance of end-to-end trained models on object recognition and image generation tasks across MNIST, ImageNet, and \\cite{majaj2015simple} datasets, particularly when labelled training data is scarce ($\<20\\%$). Our method provides a data-efficient solution for inter-modal model alignment with minimal supervision.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2505.12322>)

[PDF](<https://arxiv.org/pdf/2505.12322.pdf>)
