# Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss

Source: [Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss](<https://www.sota2.com/research/paper/68fa73571e209131cea5bf32>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:48:16.237Z

Authors: Ruijie Zheng, Yongyuan Liang, Xiyao Wang, Shuang Ma, Hal Daum\\'e III, Huazhe Xu, John Langford, Praveen Palanisamy, Kalyan Shankar Basu, Furong Huang

Published: 2024-02-09T05:04:40.000Z

## Abstract

We present Premier-TACO, a multitask feature representation learning approach designed to improve few-shot policy learning efficiency in sequential decision-making tasks. Premier-TACO leverages a subset of multitask offline datasets for pretraining a general feature representation, which captures critical environmental dynamics and is fine-tuned using minimal expert demonstrations. It advances the temporal action contrastive learning (TACO) objective, known for state-of-the-art results in visual control tasks, by incorporating a novel negative example sampling strategy. This strategy is crucial in significantly boosting TACO's computational efficiency, making large-scale multitask offline pretraining feasible. Our extensive empirical evaluation in a diverse set of continuous control benchmarks including Deepmind Control Suite, MetaWorld, and LIBERO demonstrate Premier-TACO's effectiveness in pretraining visual representations, significantly enhancing few-shot imitation learning of novel tasks. Our code, pretraining data, as well as pretrained model checkpoints will be released at https://github.com/PremierTACO/premier-taco. Our project webpage is at https://premiertaco.github.io.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Future state retrieval on DROID t+5 (2.0s prediction horizon)](<https://www.sota2.com/research/sota/future-state-retrieval-on-droid-t-plus-5-2-0s-prediction-horizon.md>): Future state retrieval; DROID t+5 (2.0s prediction horizon); Hit@1: 4.36.

- [Future state retrieval on DROID t+3 (1.2s prediction horizon)](<https://www.sota2.com/research/sota/future-state-retrieval-on-droid-t-plus-3-1-2s-prediction-horizon.md>): Future state retrieval; DROID t+3 (1.2s prediction horizon); Hit@1: 3.61.

- [Future state retrieval on DROID t+1 (0.4s prediction horizon)](<https://www.sota2.com/research/sota/future-state-retrieval-on-droid-t-plus-1-0-4s-prediction-horizon.md>): Future state retrieval; DROID t+1 (0.4s prediction horizon); Hit@1: 5.14.

- [Closed-loop Planning on RoboCasa Rc-Pl](<https://www.sota2.com/research/sota/closed-loop-planning-on-robocasa-rc-pl.md>): Closed-loop Planning; RoboCasa Rc-Pl; Success Rate: 2.6.

- [Closed-loop Planning on RoboCasa Rc-R](<https://www.sota2.com/research/sota/closed-loop-planning-on-robocasa-rc-r.md>): Closed-loop Planning; RoboCasa Rc-R; Success Rate: 3.3.

- [Offline Action Matching on DROID](<https://www.sota2.com/research/sota/offline-action-matching-on-droid.md>): Offline Action Matching; DROID; Action Score: 11.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2402.06187>)

[PDF](<https://arxiv.org/pdf/2402.06187.pdf>)
