# $\\pi$-Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation

Source: [$\\pi$-Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation](<https://www.sota2.com/research/paper/68fa6f511e209131cea44f87>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:18:28.677Z

Authors: Chengyue Wu, Teng Wang, Yixiao Ge, Zeyu Lu, Ruisong Zhou, Ying Shan, Ping Luo

Published: 2023-04-27T17:49:54.000Z

## Abstract

Foundation models have achieved great advances in multi-task learning with a unified interface of unimodal and multimodal tasks. However, the potential of such multi-task learners has not been exploited during transfer learning. In this work, we present a universal parameter-efficient transfer learning method, termed Predict-Interpolate Tuning ($\\pi$-Tuning), for vision, language, and vision-language tasks. It aggregates the parameters of lightweight task-specific experts learned from similar tasks to aid the target downstream task. The task similarities are predicted in a unified modality-independent space, yielding a scalable graph to demonstrate task relationships. $\\pi$-Tuning has several appealing benefits. First, it flexibly explores both intra- and inter-modal transferability between similar tasks to improve the accuracy and robustness of transfer learning, especially in data-scarce scenarios. Second, it offers a systematical solution for transfer learning with multi-task prediction-and-then-interpolation, compatible with diverse types of parameter-efficient experts, such as prompt and adapter. Third, an extensive study of task-level mutual benefits on 14 unimodal and 6 multimodal datasets shows that $\\pi$-Tuning surpasses fine-tuning and other parameter-efficient transfer learning methods both in full-shot and low-shot regimes. The task graph also enables an in-depth interpretable analysis of task transferability across modalities. The code will be available at https://github.com/TencentARC/pi-Tuning.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Diverse Language Understanding on 62 downstream tasks](<https://www.sota2.com/research/sota/diverse-language-understanding-on-62-downstream-tasks.md>): Diverse Language Understanding; 62 downstream tasks; Average Accuracy: 66.8.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2304.14381>)

[PDF](<https://arxiv.org/pdf/2304.14381.pdf>)