Partial Least Squares Regression (PLS)

http://faculty.chass.ncsu.edu/garson/PA765/pls.htm

Overview

• Partial least squares (PLS) is sometimes called "Projection to Latent Structures" because of its general strategy. The X variables (the predictors) are reduced to principal components, as are the Y variables (the dependents). The components of X are used to predict the scores on the Y components, and the predicted Y component scores are used to predict the actual values of the Y variables. In constructing the principal components of X, the PLS algorithm iteratively maximizes the strength of the relation of successive pairs of X and Y component scores by maximizing the covariance of each X-score with the Y variables. This strategy means that while the original X variables may be multicollinear, the X components used to predict Y will be orthogonal. Also, the X variables may have missing values, but there will be a computed score for every case on every X component. Finally, since only a few components (often two or three) will be used in predictions, PLS coefficients may be computed even when there may have been more original X variables than observations (though greater cases are recommended). In contrast, any of these three conditions (multicollinearity, missing values, and too few cases in relation to variables) may well render traditional OLS regression estimates unreliable (and estimates by other procedures in the general and generalized linear model families) .

  Partial least squares (PLS) regression/path analysis is thus an alternative to OLS regression, canonical correlation, or structural equation modeling (SEM) for analysis of systems of independent and response variables. In fact, PLS is sometimes called "component-based SEM," in contrast to the usual covariance-based structural equation modeling. PLS is a predictive technique which can handle many independent variables, even when predictors display multicollinearity. Like canonical correlation or multivariate GLM, it can also relate the set of independent variables to a set of multiple dependent (response) variables. However, PLS is less than satisfactory as an explanatory technique because it is low in power to filter out variables of minor causal importance (Tobias, 1997: 1).

  The advantages of PLS include ability to model multiple dependents as well as multiple independents; ability to handle multicollinearity among the independents; robustness in the face of data noise and missing data; and creating independent latents directly on the basis of crossproducts involving the response variable(s), making for stronger predictions. Disadvantages of PLS include greater difficulty of interpreting the loadings of the independent latent variables (which are based on crossproduct relations with the response variables, not based as in common factor analysis on covariances among the manifest independents) and because the distributional properties of estimates are not known, the researcher cannot assess significance except through bootstrap induction. Overall, the mix of advantages and disadvantages means PLS is favored as a predictive technique and not as an interpretive technique, except for exploratory analysis as a prelude to an intepretive technique such as multiple linear regression or covariance-based structural equation modeling.

  Though developed by Herman Wold (Wold, 1981, 1985) for econometrics, PLS first gained popularity in chemometric research and later industrial applications. It has since spread to research in education, marketing, and the social sciences.

  PLS may be implemented as a regression model, predicting one or more dependents from a set of one or more independents; or it can be implemented as a path model, akin to structural equation modeling. PLS is implemented as a regression model by SPSS and by SAS's PROC PLS. SmartPLS is the most prevalent implementation as a path model.