ISLR 7.7 广义可加模型
要点:
0.广义可加模型介绍
1.用于回归问题的GAM
-- 多元线性回归的推广
2.用于分类问题的GAM
-- 逻辑回归的推广
3.GAM的优点与不足
0. Generalized Additive Models
Polynomials, Step functions & Splines can be seen as extensions of simple linear regression
- flexibly predicting
on the basis of a
「Generalized Additive Models (GAMs)」 provide a general framework for extending multiple linear model
- by allowing non-linear functions for
while
1. GAMs for Regression
Recall Multiple Linear Regression:
GAMs replace each linear component
- It is called an 「addictive model」 because we calculate a separate
,
- and then add together all of them.
❝ The beauty of GAMs is that we can use different methods as building blocks for fitting an additive model
❞
Example:
To predict the wage
, fitting the model:
year
andage
are quantitative, fit both functions using natural splineseducation
is qualitative with 5 levels: <HS, HS, <Coll, Coll, >Coll, fit step function for each level via the dummy variable.
![a80f027a84a71538ea361862c0cf8acd.png](https://i-blog.csdnimg.cn/blog_migrate/4e733f79f2ed5eac0c4736c09acda360.jpeg)
Since natural splines can be constructed using an pre-chosen set of basis functions
- the entire model is just a big regreesion onto dummy and spline basis variables using least squares
![1741a5a482ba1866d3b3fc2c484dfb54.png](https://i-blog.csdnimg.cn/blog_migrate/5670ba7b8ebdc29625b8062ec59ecd54.jpeg)
Using Smooth Splines as the building blocks looks rather similar
- fitting via backfitting instead of least square
2. GAMs for Classification
Assume
- Let
Recall Logistic Regression:
- This logit is the log of the odds of
versus
, which represents as a linear function of the predictors.
Logic Regression Extension GAM for non-linear relationships:
Example:
To predict the probability that an individual's income exceeds $25,000 per year, fitting the GAM:
-
is fit via
-
is fit as a
education
![f0fa817b43fffffbfbc718cce915b42d.png](https://i-blog.csdnimg.cn/blog_migrate/05f1e35b8ad1d24a6d8e01f65a5cf185.jpeg)
- Fact: no one <HighSchool make wage>250
- CI very wide
- need refit the GAM without <HS level:
![bacbf8169732476cd7e44e4b831cc5a9.png](https://i-blog.csdnimg.cn/blog_migrate/f65c19ca25c89d2638194f4865e5bc33.jpeg)
3. Pros and Cons of GAMs
Advantages:
- GAMs allow us to fit a non-linear
to each
automatically
- standard linear regression will miss this
- The Non-linear Fits potentially make more accurate predictions for the response
- Because the model is additive, we can
- still examine the effect of each
on
- The smoothness of the function
for the variable
can be summarized via
Limitations:
- The model is restricted to be additive
- Important interactions with many variables can be missed
- need to manually add
- TOGO: For fully general models without limitations, we will look for even more flexible methods:
- Random Forest, Boosting
4. Reference
An Introduction to Statistical Learning, with applications in R (Springer, 2013)