Multiclass classification

2019独角兽企业重金招聘Python工程师标准>>>

1: Introduction To The Data

The dataset we will be working with contains information on various cars. For each car we have information about the technical aspects of the vehicle such as the motor's displacement, the weight of the car, the miles per gallon, and how fast the car accelerates. Using this information we will predict the origin of the vehicle, either North America, Europe, or Asia. We can see, that unlike our previous classification datasets, we have three categories to choose from, making our task slightly more challenging.

Here's a preview of the data:

18.0   8   307.0      130.0      3504.      12.0   70  1    "chevrolet chevelle malibu"

15.0   8   350.0      165.0      3693.      11.5   70  1    "buick skylark 320"

18.0   8   318.0      150.0      3436.      11.0   70  1    "plymouth satellite"

Here are the columns in the dataset:

• mpg -- Miles per gallon, Continuous.
• cylinders -- Number of cylinders in the motor, Integer, Ordinal, and Categorical.
• displacement -- Size of the motor, Continuous.
• horsepower -- Horsepower produced, Continuous.
• weight -- Weights of the car, Continuous.
• acceleration -- Acceleration, Continuous.
• year -- Year the car was built, Integer and Categorical.
• origin -- Integer and Categorical. 1: North America, 2: Europe, 3: Asia.
• car_name -- Name of the car.

Instructions

• Import the Pandas library and read auto.csv into a Dataframe named cars.

• Use the Series method unique to assign the unique elements in the column origin tounique_regions. Then use theprint function to displayunique_regions.

import pandas as pd
cars = pd.read_csv("auto.csv")
print(cars.head())
unique_regions=cars["origin"].unique()
print(unique_regions)

2: Dummy Variables

In previous classification missions, categorical variables have been represented in the dataset using integer values (like 0 and 1) for us already. In many cases, like with this dataset, you'll have to create numeric representation of categorical values yourself. For this dataset, categorical variables exist in three columns, cylinders, year, and origin. The cylinders and year columns must be converted to numeric values so we can use them to predict label origin. Even though the column year is a number, we’re going to treat them like categories. The year 71 is unlikely to relate to the year 70 in the same way those two numbers do numerically, but rather just as two different labels. In these instances, it is always safer to treat discrete values as categorical variables.

We must use dummy variables for columns containing categorical values. Whenever we have more than 2 categories, we need to create more columns to represent the categories. Since we have 5 different categories of cylinders, we could use 3, 4, 5, 6, and 8 to represent the different categories. We can split the column into separate binary columns:

• cyl_3 -- Does the car have 3 cylinders? 0 if False, 1 if True.
• cyl_4 -- Does the car have 4 cylinders? 0 if False, 1 if True.
• cyl_5 -- Does the car have 5 cylinders? 0 if False, 1 if True.
• cyl_6 -- Does the car have 6 cylinders? 0 if False, 1 if True.
• cyl_8 --Does the car have 8 cylinders? 0 if False, 1 if True.

We can use the Pandas function get_dummies to return a Dataframe containing binary columns from the values in the cylinderscolumn. In addition, if we set the prefix parameter to cyl, Pandas will pre-pend the column names to match the style we'd like:

dummy_df = pd.get_dummies(cars["cylinders"], prefix="cyl")

We then use the Pandas function concat to add the columns from this Dataframe back to cars:

cars = pd.concat([cars, dummy_df], axis=1)

Now it's your turn! Repeat the same process for the year column.

Instructions

• Use the Pandas functionget_dummies to create dummy values from the year column.

  • Use the prefix attribute to prepend year to each of the resulting column names.
  • Assign the resulting Dataframe todummy_years.

• Use the Pandas method concatto concatenate the columns fromdummy_years to cars.

• Use the Dataframe method dropto drop the years andcylinders columns from cars.

• Display the first 5 rows of the newcars Dataframe to confirm.

dummy_cylinders = pd.get_dummies(cars["cylinders"], prefix="cyl")
cars = pd.concat([cars, dummy_cylinders], axis=1)
print(cars.head())
dummy_years=pd.get_dummies(cars["year"],prefix="year")
cars=pd.concat([cars,dummy_years],axis=1)
cars=cars.drop("cylinders",axis=1)
cars=cars.drop("year",axis=1)
print(dummy_years.head())

 

3: Multiclass Classification

In previous missions, we explored binary classification, where there were only 2 possible categories, or classes. When we have 3 or more categories, we call the problem a multiclass classification problem. There are a few different methods of doing multiclass classification and in this mission, we'll focus on the one-versus-all method.

The one-versus-all method is a technique where we choose a single category as the Positive case and group the rest of the categories as the False case. We're essentially splitting the problem into multiple binary classification problems. For each observation, the model will then output the probability of belonging to each category.

To start let's split our data into a training and test set. We've randomized the cars Dataframe for you already to start things off and assigned the shuffled Dataframe to shuffled_cars.

Instructions

• Split the shuffled_carsDataframe into 2 Dataframes:train and test.
  • Assign the first 70% of theshuffled_cars to train.
  • Assign the last 30% of theshuffled_cars to test.

shuffled_rows = np.random.permutation(cars.index)
shuffled_cars = cars.iloc[shuffled_rows]
highest_train_row=int(cars.shape[0]*.7)
train=shuffled_cars.iloc[0:highest_train_row]
test=shuffled_cars.iloc[highest_train_row:]

4: Training A Multiclass Logistic Regression Model

In the one-vs-all approach, we're essentially converting an n-class (in our case n is 3) classification problem into n binary classification problems. For our case, we'll need to train 3 models:

• A model where all cars built in North America are considered Positive (1) and those built in Europe and Asia are considered Negative (0).
• A model where all cars built in Europe are considered Positive (1) and those built in North America and Asia are considered Negative (0).
• A model where all cars built in Asia are labeled Positive (1) and those built in North America and Europe are considered Negative (0).

Each of these models is a binary classification model that will return a probability between 0 and 1. When we apply this model on new data, a probability value will be returned from each model (3 total). For each observation, we choose the label corresponding to the model that predicted the highest probability.

We'll use the dummy variables we created from the cylinders and year columns to train 3 models using the LogisticRegression class from scikit-learn.

Instructions

For each value in unique_origins, train a logistic regression model with the following parameters (Documentation):

• X: Dataframe containing just the cylinder & year binary columns.
• y: list (or Series) of Boolean values:
  • True if observation's value for origin matches the current iterator variable.
  • False if observation's value for origin doesn't match the current iterator variable.

Add each model to the modelsdictionary with the following structure:

• key: origin value (1, 2, or 3),
• value: relevant LogistcRegression model instance.

 

from sklearn.linear_model import LogisticRegression
unique_origins = cars["origin"].unique()
unique_origins.sort()
models = {}
feater=[c for c in train.columns if c.startswith("cyl") or c.startswith("year")]

for origin in unique_origins:
    model=LogisticRegression()
    X_train=train[feater]
    y_train=train["origin"]==origin
    model.fit(X_train,y_train)
    models[origin]=model
    

5: Testing The Models

Now that we have a model for each category, we can run our test dataset through the models and evaluate how well they performed.

Instructions

• For each origin value fromunique_origins:

  • Use the LogisticRegressionpredict_proba function to return the 3 lists of predicted probabilities for the test set and add to thetesting_probsDataframe.

• Here's how the final Dataframe should look like (without all zeroes of course!):

 

	1	2	3
0	0.000	0.000	0.000
1	0.000	0.000	0.000
2	0.000	0.000	0.000
3	0.000	0.000	0.000
4	0.000	0.000	0.000
5	0.000	0.000	0.000

 

testing_probs = pd.DataFrame(columns=unique_origins)
for origin in unique_origins:
    X_test=test[features]
    testing_probs[origin]=models[origin].predict_proba(X_test)[:,1]

 

6: Choose The Origin

Now that we trained the models and computed the probabilities in each origin we can classify each observation. To classify each observation we want to select the origin with the highest probability of classification for that observation.

While each column in our dataframe testing_probs represents an origin we just need to choose the one with the largest probability. We can use the Dataframe method .idxmax() to return a Series where each value corresponds to the column or where the maximum value occurs for that observation. We need to make sure to set the axis paramater to 1 since we want to calculate the maximum value across columns. Since each column maps directly to an origin the resulting Series will be the classification from our model.

Instructions

• Classify each observation in the test set using the testing_probsDataframe.
• Assign the predicted origins topredicted_origins and use theprint function to display it.

 

predicted_origins=testing_probs.idxmax(axis=1)
print(predicted_origins)

7: Conclusion

In this mission, we learned the basics of extending logistic regression to work for multi-class classification problems. In the next mission, we'll dive into more intermediate linear regression concepts.

 

转载于:https://my.oschina.net/Bettyty/blog/752116