Basic Data Exploration(2)

1—Selecting Data for Modeling

Your dataset has so many variables to wrap your head around🤷🏼, or even to output it nicely🥀. How can you pare down that overwhelming amount of data to something you can understand🧐?

Let's start by selecting some variables using our intuition🤓.

To choose variables/columns, we’ll need to look at a list of all the columns🤯 in the dataset. This is done with the columns property of the DataFrame👇🏻:

#To choose variables/columns,
#we’ll need to look at a list of all the columns in the dataset.
import pandas as pd
melbourne_file_path = '/Users/mac/Desktop/melb_data.csv'
melbourne_data = pd.read_csv(melbourne_file_path) 
melbourne_data.columns

Out[1]: 
Index(['Suburb', 'Address', 'Rooms', 'Type', 'Price', 'Method', 'SellerG',
       'Date', 'Distance', 'Postcode', 'Bedroom2', 'Bathroom', 'Car',
       'Landsize', 'BuildingArea', 'YearBuilt', 'CouncilArea', 'Lattitude',
       'Longtitude', 'Regionname', 'Propertycount'],
      dtype='object')

2—Selecting The Prediction Target

We can pull out a variable with dot-notation. Store this single column in a Series, which is broadly like a DataFrame with only a single column of data.

Now, instead of choosing variables intuitively, but using what you're going to predict💁🏼‍♀️. The variable we pull out is called the prediction target. By convention, the prediction target is called y.

#Selecting the prediction target: y = 'Price'
y = melbourne_data.Price

（Interesting part😄）

3—Choosing "Features"

In the output above, columns other than the ‘Price' are called “features”. By convention, the features are called X.

(In this case, those columns are also inputted into our model and used to determine the home price.)(Sometimes, we will use all columns except the target as features. Other times you'll be better off with fewer features.)

#For now, we'll build a model with only a few features
melbourne_features = ['Rooms', 'Bathroom', 'Landsize', 
                      'Lattitude', 'Longtitude']
X = melbourne_data[melbourne_features]

- Is this the end of it? -No😂

Let's quickly review the data we'll be using to predict house prices using the describe method and the head method......

the describe method👀: 

#To choose variables/columns,
#we’ll need to look at a list of all the columns in the dataset.
import pandas as pd
melbourne_file_path = '/Users/mac/Desktop/melb_data.csv'
melbourne_data = pd.read_csv(melbourne_file_path) 
melbourne_data.columns
#Selecting the prediction target: y = 'Price'
y = melbourne_data.Price
#For now, we'll build a model with only a few features
melbourne_features = ['Rooms', 'Bathroom', 'Landsize', 
                      'Lattitude', 'Longtitude']
X = melbourne_data[melbourne_features]
#Predict house prices using the describe method
X.describe()

Out[2]: 
              Rooms      Bathroom       Landsize     Lattitude    Longtitude
count  13580.000000  13580.000000   13580.000000  13580.000000  13580.000000
mean       2.937997      1.534242     558.416127    -37.809203    144.995216
std        0.955748      0.691712    3990.669241      0.079260      0.103916
min        1.000000      0.000000       0.000000    -38.182550    144.431810
25%        2.000000      1.000000     177.000000    -37.856822    144.929600
50%        3.000000      1.000000     440.000000    -37.802355    145.000100
75%        3.000000      2.000000     651.000000    -37.756400    145.058305
max       10.000000      8.000000  433014.000000    -37.408530    145.526350

the head method👀: 

#To choose variables/columns,
#we’ll need to look at a list of all the columns in the dataset.
import pandas as pd
melbourne_file_path = '/Users/mac/Desktop/melb_data.csv'
melbourne_data = pd.read_csv(melbourne_file_path) 
melbourne_data.columns
#Selecting the prediction target: y = 'Price'
y = melbourne_data.Price
#For now, we'll build a model with only a few features
melbourne_features = ['Rooms', 'Bathroom', 'Landsize', 
                      'Lattitude', 'Longtitude']
X = melbourne_data[melbourne_features]
#Predict house prices using the head method
X.head()

Out[3]: 
   Rooms  Bathroom  Landsize  Lattitude  Longtitude
0      2       1.0     202.0   -37.7996    144.9984
1      2       1.0     156.0   -37.8079    144.9934
2      3       2.0     134.0   -37.8093    144.9944
3      3       2.0      94.0   -37.7969    144.9969
4      4       1.0     120.0   -37.8072    144.9941

Using these commands to visually check data is an important part of the data effort. I think we'll find some surprises💡 in the dataset that are worth checking out.