Adaptive and Robust Multi-Task Learning

About

We study the multi-task learning problem that aims to simultaneously analyze multiple datasets collected from different sources and learn one model for each of them. We propose a family of adaptive methods that automatically utilize possible similarities among those tasks while carefully handling their differences. We derive sharp statistical guarantees for the methods and prove their robustness against outlier tasks. Numerical experiments on synthetic and real datasets demonstrate the efficacy of our new methods.

Yaqi Duan, Kaizheng Wang · 2022

Related benchmarks

Task	Dataset	Result
Linear regression	Synthetic Linear Regression (n=50, d=50, K=40, ε=0.2) (train/test)	Average Local Error0.389	204
Linear regression	Synthetic Multi-task Linear Regression (n=d=50, K=40, epsilon=0.2)	Global Error (L2-norm)0.39	192
Multi-task Linear Regression (Local Parameter Estimation)	Synthetic Multi-task Linear Regression (n=d=50, ε=0.2) (test)	Local L2 Error1.173	102
Local prediction	HAR (20% train)	Local Prediction Error3.1	102
Local prediction error	HAR 50% (train)	Error Rate2	102
Local prediction	HAR 60% (train)	Local Prediction Error1.9	102
Linear regression	Synthetic Multi-task Dataset d=50, K=20, epsilon=0.2 (test)	Local Error1.091	102
Multi-task Logistic Regression	Synthetic Multi-task Logistic Regression (n=d=50, epsilon=0.2)	Local Error2.695	102
Logistic Regression	Logistic regression d=50, K=20, epsilon=0.2 varying per-task sample size n (test)	Local Error2.726	102
Linear regression	Linear Regression (synthetic)	Local Error0.984	101