California Housing
I select California Housing dataset for data analysis.
Part 1 Load Datasets
I download California Housing dataset from https://www.kaggle.com/camnugent/california-housing-prices and save it as housing.csv.
Then,I load it.
from sklearn import datasets
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import Imputer
import numpy as np
housing_data = pd.read_csv('housing.csv')
Then,split train data and test data
from sklearn.model_selection import train_test_split
train_set, test_set = train_test_split(housing_data, test_size=0.2, random_state=42)
Part 2 Explore the dataset
I explore the dataset using the histogram.
First,we need to copy the train data
housing = train_set.copy()
start to explore the dataset
housing.hist(bins=50,figsize=(20,15))
plt.show()
From the picture we can see all the features.
Part 3 Analyse the dataset
I display data through visualization using matplotlib.
Let’s look at the impact of longitude and latitude on house prices.
plt.scatter(x = housing['longitude'],y = housing['latitude'],alpha = 0.1)
plt.show()
Let’s see other features.
housing.plot(kind="scatter",x="longitude",y="latitude",alpha=0.4,
s=housing["population"]/100,label="population",
c="median_house_value",cmap=plt.get_cmap("jet"),colorbar=True,sharex=False)
plt.legend()
plt.show()
Part 4 Extract the important features
Let’s look at correlation between each pair of feature.
corr_matrix = housing.corr()
print(corr_matrix['median_house_value'].sort_values(ascending = False))
median_house_value 1.000000
median_income 0.688075
total_rooms