# Using Multiple Instance Learning to Build Multimodal Representations Source: [Using Multiple Instance Learning to Build Multimodal Representations]() Collection index: [llms.txt]() Data updated: 2025-10-26T01:24:25.039Z Authors: Peiqi Wang, William M. Wells, Seth Berkowitz, Steven Horng, Polina Golland Published: 2022-12-11T18:01:11.000Z ## Abstract Image-text multimodal representation learning aligns data across modalities and enables important medical applications, e.g., image classification, visual grounding, and cross-modal retrieval. In this work, we establish a connection between multimodal representation learning and multiple instance learning. Based on this connection, we propose a generic framework for constructing permutation-invariant score functions with many existing multimodal representation learning approaches as special cases. Furthermore, we use the framework to derive a novel contrastive learning approach and demonstrate that our method achieves state-of-the-art results in several downstream tasks. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Stenosis Classification on internal 3D heart CT dataset](): Stenosis Classification; internal 3D heart CT dataset; Accuracy: 80.56. - [Calcification Classification on internal 3D heart CT dataset](): Calcification Classification; internal 3D heart CT dataset; Accuracy: 51.39. ## Original sources and implementation [Original paper]() [PDF]()