＜机器学习实战＞第一章的 oecd_bli = oecd_bli[oecd_bli[“INEQUALITY“]==“TOT“ 什么意思？

在《机器学习实战》这本书里第一章有个实例：联合国统计的人均GDP与国民幸福度之间的关系。书中使用matlablib做散点图。大致的代码如下：

def prepare_country_stats(oecd_bli, gdp_per_capita):
    oecd_bli = oecd_bli[oecd_bli["INEQUALITY"]=="TOT"]
    oecd_bli = oecd_bli.pivot(index="Country", columns="Indicator", values="Value")
    gdp_per_capita.rename(columns={"2015": "GDP per capita"}, inplace=True)
    gdp_per_capita.set_index("Country", inplace=True)
    full_country_stats = pd.merge(left=oecd_bli, right=gdp_per_capita,
                                  left_index=True, right_index=True)
    full_country_stats.sort_values(by="GDP per capita", inplace=True)
    remove_indices = [0, 1, 6, 8, 33, 34, 35]
    keep_indices = list(set(range(36)) - set(remove_indices))
    return full_country_stats[["GDP per capita", 'Life satisfaction']].iloc[keep_indices]

# To plot pretty figures directly within Jupyter
%matplotlib inline
import matplotlib as mpl
mpl.rc('axes', labelsize=14)
mpl.rc('xtick', labelsize=12)
mpl.rc('ytick', labelsize=12)

# Download the data
import urllib.request
DOWNLOAD_ROOT = "https://raw.githubusercontent.com/ageron/handson-ml2/master/"
os.makedirs(datapath, exist_ok=True)
for filename in ("oecd_bli_2015.csv", "gdp_per_capita.csv"):
    print("Downloading", filename)
    url = DOWNLOAD_ROOT + "datasets/lifesat/" + filename
    urllib.request.urlretrieve(url, datapath + filename)

# Code example
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import sklearn.linear_model

# Load the data
oecd_bli = pd.read_csv(datapath + "oecd_bli_2015.csv", thousands=',')
gdp_per_capita = pd.read_csv(datapath + "gdp_per_capita.csv",thousands=',',delimiter='\t',
                             encoding='latin1', na_values="n/a")

# Prepare the data
country_stats = prepare_country_stats(oecd_bli, gdp_per_capita)
X = np.c_[country_stats["GDP per capita"]]
y = np.c_[country_stats["Life satisfaction"]]

# Visualize the data
country_stats.plot(kind='scatter', x="GDP per capita", y='Life satisfaction')
plt.show()

# Select a linear model
model = sklearn.linear_model.LinearRegression()

# Train the model
model.fit(X, y)

# Make a prediction for Cyprus
X_new = [[22587]]  # Cyprus' GDP per capita
print(model.predict(X_new)) # outputs [[ 5.96242338]]

有用户提问： oecd_bli = oecd_bli[oecd_bli["INEQUALITY"]=="TOT"] 是什么？

编者回答：This syntax is a special indexing syntax that works with Pandas DataFrames, NumPy arrays, TensorFlow tensors and a few other libraries.

oecd_bli = oecd_bli[oecd_bli["INEQUALITY"]=="TOT"]

首先注意 oecd_bli["INEQUALITY"]=="TOT" 是一个 pandas Series ，只要 "INEQUALITY" 的feature是 "TOT"的时候它等于布尔运算符 True

所以oecd_bli[oecd_bli["INEQUALITY"]=="TOT"] 是一个新的pandas DataFrame，其内容是上面这个True对应位置的样本。

即：

import pandas as pd

oecd_bli = pd.DataFrame({
    "INEQUALITY": ["a", "b", "TOT", "TOT", "c", "TOT"],
    "Other": [10, 20, 30, 40, 50, 60]
})

print(oecd_bli["INEQUALITY"]=="TOT")
# prints this Pandas Series:
# 0    False
# 1    False
# 2     True 
# 3     True
# 4    False
# 5     True
# Name: INEQUALITY, dtype: bool

oecd_bli = oecd_bli[oecd_bli["INEQUALITY"]=="TOT"]

print(oecd_bli)
# prints:
#  INEQUALITY  Other
# 2        TOT      30
# 3        TOT      40
# 5        TOT      60