kaggle教程--Introduction to ML

Using Pandas to Get Familiar With Your Data

The first step in any machine learning project is familiarize yourself with the data. You’ll use the Pandas library for this. Pandas is the primary tool data scientists use for exploring and manipulating data. Most people abbreviate pandas in their code as pd. We do this with the command

# read data
data = pd.read_csv(path)
data.describe()

We’ll start by picking a few variables using our intuition. Later courses will show you statistical techniques to automatically prioritize variables.

# show columns 
melbourne_data.columns
# drop the NAN instance
melbourne_data = melbourne_data.dropna(axis=0)

使用点来选target列， column list选特征列

1. Dot notation, which we use to select the “prediction target”
2. Selecting with a column list, which we use to select the “features”

y = melbourne_data.Price
melbourne_features = ['Rooms', 'Bathroom', 'Landsize', 'Lattitude', 'Longtitude']
X = melbourne_data[melbourne_features]
X.describe()
X.head()

Building Your Model

The steps to building and using a model are:

• Define: What type of model will it be? A decision tree? Some other type of model? Some other parameters of the model type are specified too.
• Fit: Capture patterns from provided data. This is the heart of modeling.
• Predict: Just what it sounds like
• Evaluate: Determine how accurate the model’s predictions are.

from sklearn.tree import DecisionTreeRegressor

# Define model. Specify a number for random_state to ensure same results each run
melbourne_model = DecisionTreeRegressor(random_state=1)

# Fit model
melbourne_model.fit(X, y)

predictions = melbourne_model.predict(X)
print(predictions)

Specifying a number for random_stateensures you get the same results in each run.

Model Validation

use model validation to measure the quality of your model. Measuring model quality is the key to iteratively improving your models.

这个模型我们使用 MAE作为评价准则

from sklearn.metrics import mean_absolute_error

predicted_home_prices = melbourne_model.predict(X)
mean_absolute_error(y, predicted_home_prices)

划分训练集和测试集 train_test_split

from sklearn.model_selection import train_test_split
# split data into training and validation data, for both features and target
# The split is based on a random number generator. Supplying a numeric value to
# the random_state argument guarantees we get the same split every time we
# run this script.
train_X, val_X, train_y, val_y = train_test_split(X, y, random_state = 0)
# Define model
melbourne_model = DecisionTreeRegressor()
# Fit model
melbourne_model.fit(train_X, train_y)

# get predicted prices on validation data
val_predictions = melbourne_model.predict(val_X)
print(mean_absolute_error(val_y, val_predictions))

Experimenting With Different Models

overfitting, where a model matches the training data almost perfectly, but does poorly in validation and other new data.

When a model fails to capture important distinctions and patterns in the data, so it performs poorly even in training data, that is called underfitting.

the max_leaf_nodes argument provides a very sensible way to control overfitting vs underfitting. The more leaves we allow the model to make, the more we move from the underfitting area in the above graph to the overfitting area.

We can use a utility function to help compare MAE scores from different values for max_leaf_nodes:

from sklearn.metrics import mean_absolute_error
from sklearn.tree import DecisionTreeRegressor

def get_mae(max_leaf_nodes, train_X, val_X, train_y, val_y):
    model = DecisionTreeRegressor(max_leaf_nodes=max_leaf_nodes, random_state=0)
    model.fit(train_X, train_y)
    preds_val = model.predict(val_X)
    mae = mean_absolute_error(val_y, preds_val)
    return(mae)

# compare MAE with differing values of max_leaf_nodes
for max_leaf_nodes in [5, 50, 500, 5000]:
    my_mae = get_mae(max_leaf_nodes, train_X, val_X, train_y, val_y)
    print("Max leaf nodes: %d  \t\t Mean Absolute Error:  %d" %(max_leaf_nodes, my_mae))

random forest 一般用随机森林可以取得一个不错的成绩

The random forest uses many trees, and it makes a prediction by averaging the predictions of each component tree. It generally has much better predictive accuracy than a single decision tree and it works well with default parameters. If you keep modeling, you can learn more models with even better performance, but many of those are sensitive to getting the right parameters.

from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_absolute_error

forest_model = RandomForestRegressor(random_state=1)
forest_model.fit(train_X, train_y)
melb_preds = forest_model.predict(val_X)
print(mean_absolute_error(val_y, melb_preds))

But one of the best features of Random Forest models is that they generally work reasonably even without this tuning.

You’ll soon learn the XGBoost model, which provides better performance when tuned well with the right parameters (but which requires some skill to get the right model parameters).

提交结果

# To improve accuracy, create a new Random Forest model which you will train on all training data
rf_model_on_full_data = RandomForestRegressor(random_state=1)

# fit rf_model_on_full_data on all data from the training data
rf_model_on_full_data.fit(X,y)

# path to file you will use for predictions
test_data_path = '../input/test.csv'

# read test data file using pandas
test_data = pd.read_csv(test_data_path)

# create test_X which comes from test_data but includes only the columns you used for prediction.
# The list of columns is stored in a variable called features
test_X = test_data[features]

# make predictions which we will submit. 
test_preds = rf_model_on_full_data.predict(test_X)

# The lines below shows how to save predictions in format used for competition scoring
# Just uncomment them.

output = pd.DataFrame({'Id': test_data.Id,
                       'SalePrice': test_preds})
output.to_csv('submission.csv', index=False)

1. Click the Commit button.
2. After your code has finished running, click the “Open Version” button. This brings you into the “viewer mode” for your notebook. You will need to scroll down to get back to these instructions.
3. Click Output button on the left of your screen.

This will bring you to a part of the screen that looks like this:

Select the button to submit and you will see your score. You have now successfully submitted to the competition.

4. If you want to keep working to improve your model, select the edit button. Then you can change your model and repeat the process to submit again. There’s a lot of room to improve your model, and you will climb up the leaderboard as you work.

There are many ways to improve your model, and experimenting is a great way to learn at this point.

The best way to improve your model is to add features. Look at the list of columns and think about what might affect home prices. Some features will cause errors because of issues like missing values or non-numeric data types.