Bobo老师机器学习笔记-数据归一化

最新推荐文章于 2021-01-11 15:58:14 发布

风中静行

最新推荐文章于 2021-01-11 15:58:14 发布

阅读量525

点赞数

分类专栏：机器学习文章标签：机器学习数据归一化

本文链接：https://blog.csdn.net/sxb0841901116/article/details/83058483

版权

机器学习专栏收录该内容

40 篇文章 11 订阅

订阅专栏

实现算法：


def normalizate_max_min(X):
    """
    利用最大和最小化方式进行归一化，过一化的数据集中在【0， 1】
    :param X:
    :return:
    """
    np.asarray(X, dtype=float)
    if len(X.shape) == 1:
        normalizate_array = ( X - np.min(X) ) / (np.max(X) - np.min(X))
    else:
        normalizate_array = np.zeros(X.shape)
        for column in range(X.shape[1]):
            normalizate_array[:, column] = (X[:, column] - np.min(X[:, column])) / (np.max(X[:, column] - np.min(X[:, column])))

    return normalizate_array


def standardization(X):
    """
    利用z-scores实现，标准化后的数据大概在【-1.5到1.5】，数据的平均数为0，方差为1
    :param X: 可以是矩阵
    :return:
    """

    np.asarray(X, dtype=float)
    if len(X.shape) == 1:
        return (X - np.mean(X)) / np.std(X)
    else:
        dt = np.zeros(X.shape)
        for i in range(X.shape[1]):
            dt[:, i] = (X[:,i] - np.mean(X)) / np.std(X[:, ])

        return dt

 Y = np.random.randint(1, 10, (20, 5))
    nml_data = normalizate_max_min(Y)
    std_data = standardization(Y)

    plt.scatter(Y[:, 0], Y[:, 1], color='green', label='Original Data')
    plt.scatter(std_data[:, 0], std_data[:, 1], color='red', label="Max-Min normalization")
    plt.scatter(nml_data[:, 0], nml_data[:, 1], color='blue', marker='+', label="Standardazation")
    plt.legend()
    plt.show()

把测试数据进行图形展示：

Sklearn中实现可以详细可以看下面链接：

http://cwiki.apachecn.org/pages/viewpage.action?pageId=10814134

主要有一个模块交preprocessing,里面有实现各种scaler。

比如下面两个例子：

    from sklearn.preprocessing import MinMaxScaler, StandardScaler, scale
    max_min_scaler = MinMaxScaler()
    x_train_data = max_min_scaler.fit_transform(Y)
    print(x_train_data)

    standard_scaler =  StandardScaler()
    x_train_data = standard_scaler.fit_transform(Y)
    print(x_train_data)

用KNN算法测试：

from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target)

standardscaler = StandardScaler()
standardscaler.fit(X_train)

X_train_stanard = standardscaler.transform(X_train)
X_test_standard = standardscaler.transform(X_test)

knn_clf = KNeighborsClassifier(n_neighbors=3)
knn_clf.fit(X_train_stanard, y_train)

knn_clf.predict(X_test_standard)
print(knn_clf.score(X_test_standard, y_test))