1—Using Pandas to Get Familiar With Your Data
The first step in any machine learning project is familiarize yourself with the data. (Pandas is the primary tool data scientists use for exploring and manipulating data. Most people abbreviate pandas in their code as pd. We'll do this with the command.)
import pandas as pd
Let’s load and explore the data about home prices in Melbourne, Australia with the following commands.In the hands-on exercises, I will apply the same processes to a new dataset, maybe the home-prices data from Iowa or Florida.
Input:
import pandas as pd
melbourne_file_path = '/Users/mac/Desktop/melb_data.csv'
# read the data and store data in DataFrame titled melbourne_data
melbourne_data = pd.read_csv(melbourne_file_path)
# print a summary of the data in Melbourne data
melbourne_data.describe()
Output:
Out[5]:
Rooms Price ... Longtitude Propertycount
count 13580.000000 1.358000e+04 ... 13580.000000 13580.000000
mean 2.937997 1.075684e+06 ... 144.995216 7454.417378
std 0.955748 6.393107e+05 ... 0.103916 4378.581772
min 1.000000 8.500000e+04 ... 144.431810 249.000000
25% 2.000000 6.500000e+05 ... 144.929600 4380.000000
50% 3.000000 9.030000e+05 ... 145.000100 6555.000000
75% 3.000000 1.330000e+06 ... 145.058305 10331.000000
max 10.000000 9.000000e+06 ... 145.526350 21650.000000
[8 rows x 13 columns]
2—Interpreting Data Description
The results show 8 numbers for each column in your original dataset.
count: Shows how many rows have non-missing values.
(Yes, the count shows the total number of non-missing values, which means there are missing values in the dataset because the size of the 2nd bedroom wouldn't be collected when surveying a 1 bedroom house. )
mean: Which is the average.
std: Std is the standard deviation, which measures how numerically spread out the values are.
min: The first (smallest) value is the min.
(Q: Quartile)
Q1-25%: If you go 25% way through the list, you'll find a number that is bigger than 25% of the values and smaller than 75% of the values. (pronounced "25th percentile")
(For example, if I score 40, the percentage of 40 is 25% , which means that 75% of the students score above 40 and 25% score below 40.)
Q2-50%: If you go 50% way through the list, you'll find a number that is bigger than 50% of the values and smaller than 50% of the values.
Q3-75%: If you go 75% way through the list, you'll find a number that is bigger than 75% of the values and smaller than 25% of the values.
Max: The largest number.