Least Angle Regression

About

The purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be applied. Typically we have available a large collection of possible covariates from which we hope to select a parsimonious set for the efficient prediction of a response variable. Least Angle Regression (LARS), a new model selection algorithm, is a useful and less greedy version of traditional forward selection methods. Three main properties are derived: (1) A simple modification of the LARS algorithm implements the Lasso, an attractive version of ordinary least squares that constrains the sum of the absolute regression coefficients; the LARS modification calculates all possible Lasso estimates for a given problem, using an order of magnitude less computer time than previous methods. (2) A different LARS modification efficiently implements Forward Stagewise linear regression, another promising new model selection method;

Bradley Efron, Trevor Hastie, Iain Johnstone, Robert Tibshirani · 2004

Related benchmarks

Task	Dataset	Result
Classification	ELECTRICITY (10-fold cross-validation)	Test F1 Score60.7	7
Classification	GAS-DRIFT (10-fold cross-validation)	Test F1 Score0.826	7
Classification	LETTER (10-fold cross-val)	Test F1 Score23.8	7
Classification	MOZILLA4 (10-fold cross-val)	Test F1 Score62	7
Classification	MNIST (10-fold cross-validation)	Test F1 Score46.8	7
Classification	AMAZON-EMPLOYEE-ACCESS (10-fold cross-validation)	Test F1 Score0.812	7
Classification	COVERTYPE (10-fold cross-validation)	Test F147.6	7
Regression	42 Regression Datasets (840 train test)	Relative RMSE (50% Quantile)1.196	6
L1 regularized least squares	Example 5.3	Time Complexity2	2