Strong rules for discarding predictors in lasso-type problems

About

We consider rules for discarding predictors in lasso regression and related problems, for computational efficiency. El Ghaoui et al (2010) propose "SAFE" rules that guarantee that a coefficient will be zero in the solution, based on the inner products of each predictor with the outcome. In this paper we propose strong rules that are not foolproof but rarely fail in practice. These can be complemented with simple checks of the Karush- Kuhn-Tucker (KKT) conditions to provide safe rules that offer substantial speed and space savings in a variety of statistical convex optimization problems.

Robert Tibshirani, Jacob Bien, Jerome Friedman, Trevor Hastie, Noah Simon, Jonathan Taylor, Ryan J. Tibshirani · 2010

Related benchmarks

Task	Dataset	Result
Lasso Optimization	GENE	Average Computing Time1.13	6
Lasso Optimization	NYT	Average Computing Time33.64	6
Lasso Optimization	MNIST	Average Computing Time5.58	6
Lasso Optimization	GWAS	Average Computing Time21.89	6
Group Lasso	GRVS	Time3.3	5
Group Lasso	GENE-SPLINE	Time6.34	5