Next Project: Regularized Logistic Regression

L1 Regularized Logistic Regression effectively handles large number of predictors and serves variable selection simultaneously. [1] indicates that L1 RLR can be implemented via IRLS-LARS algorithm. You can tweak PROC GLMSELECT in v9.2 for this.

L2 Reguarlized Logistic Regression can be used to approximate SVM solutions [2], and can be implemented via TR-IRLS as suggested by [3], which is a ridge LR.

Reference:
[1]Su-In Lee, Honglak Lee, Pieter Abbeel and Andrew Y. Ng, "Efficient L1 Regularized Logistic Regression", Proceedings of Annual Conference of American Association for Artificial Intelligence, 2006

[2] Jian Zhang, Rong Jin, Yiming Yang & Alex G. Hauptmann, "Modified Logistic Regression: An Approximation to SVM and Its Applications in Large-Scale Text Categorization", Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003), Washington DC, 2003.

[3]Paul Komarek, "Logistic Regression for Data Mining and High-Dimensional Classification", Ph.D Dissertation, Robotics Institute, Carnegie Mellon University, 2004