%matplotlib inline
import random
import numpy as np
import scipy as sp
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import statsmodels.api as sm
import statsmodels.formula.api as smf
sns.set_context("talk")
Anscombe's quartet
Anscombe's quartet comprises of four datasets, and is rather famous. Why? You'll find out in this exercise.
anascombe = pd.read_csv('data/anscombe.csv')
anascombe.head()
dataset x y
0 I 10 8.04
1 I 8 6.95
2 I 13 7.58
3 I 9 8.81
4 I 11 8.33
Part 1
For each of the four datasets...
- Compute the mean and variance of both x and y
- Compute the correlation coefficient between x and y
- Compute the linear regression line: y=β0+β1x+ϵy=β0+β1x+ϵ (hint: use statsmodels and look at the Statsmodels notebook)
代码如下:
#Part 1
print("mean:")
print(anascombe.groupby('dataset')['x','y'].mean())
print("variance:")
print(anascombe.groupby('dataset')['x', 'y'].var())
print("the correlation coefficient between x and y are:")
print(anascombe.groupby('dataset').corr())
line1 = smf.ols('y ~ x', anascombe.groupby('dataset').get_group('I')).fit()
print("dataset1:")
print(line1.params)
line2 = smf.ols('y ~ x', anascombe.groupby('dataset').get_group('II')).fit()
print("dataset2:")
print(line2.params)
line3 = smf.ols('y ~ x', anascombe.groupby('dataset').get_group('III')).fit()
print("dataset3:")
print(line3.params)
line4 = smf.ols('y ~ x', anascombe.groupby('dataset').get_group('IV')).fit()
print("dataset4:")
print(line4.params)
输出结果如下:
mean:
x y
dataset
I 9.0 7.500909
II 9.0 7.500909
III 9.0 7.500000
IV 9.0 7.500909
variance:
x y
dataset
I 11.0 4.127269
II 11.0 4.127629
III 11.0 4.122620
IV 11.0 4.123249
the correlation coefficient between x and y are:
x y
dataset
I x 1.000000 0.816421
y 0.816421 1.000000
II x 1.000000 0.816237
y 0.816237 1.000000
III x 1.000000 0.816287
y 0.816287 1.000000
IV x 1.000000 0.816521
y 0.816521 1.000000
dataset1:
Intercept 3.000091
x 0.500091
dtype: float64
dataset2:
Intercept 3.000909
x 0.500000
dtype: float64
dataset3:
Intercept 3.002455
x 0.499727
dtype: float64
dataset4:
Intercept 3.001727
x 0.499909
dtype: float64
Part 2
Using Seaborn, visualize all four datasets.
hint: use sns.FacetGrid combined with plt.scatter
代码如下:
#Part 2
x=sns.FacetGrid(anascombe,col='dataset')
x.map(plt.scatter,"x","y")
plt.show()
输出结果如下: