机器学习-sklearn模块数据预处理

1.数据标准化，使数据满足高斯分布
preprocessing.scale()函数

import numpy as np
from sklearn import preprocessing
from scipy.stats import anderson

rain = np.load('rain.npy')
rain = .1 * rain
rain[rain < 0] = 0.05/2

scaled = preprocessing.scale(rain)
print("rain mean",scaled.mean())
print("rain variance",scaled.var())
print("anderson rain",anderson(scaled))

2.使数据所有样本数值缩放到（-1，1）之间
方法一：

from sklearn import preprocessing

X = [[ 1., -1.,  2.],
     [ 2.,  0.,  0.],
     [ 0.,  1., -1.]]
X_normalized = preprocessing.normalize(X, norm='l2')

>>> X_normalized                                      
array([[ 0.40..., -0.40...,  0.81...],
       [ 1.  ...,  0.  ...,  0.  ...],
       [ 0.  ...,  0.70..., -0.70...]])

方法二：

XY = [[ 1., -1.,  2.],
     [ 2.,  0.,  0.],
      [ 0.,  1., -1.]]
normalizer = preprocessing.Normalizer().fit(XY)
print(normalizer.transform(XY))

3.二值化数据

>>> X = [[ 1., -1.,  2.],
...      [ 2.,  0.,  0.],
...      [ 0.,  1., -1.]]

>>> binarizer = preprocessing.Binarizer().fit(X)  # fit does nothing
>>> binarizer
Binarizer(copy=True, threshold=0.0)

>>> binarizer.transform(X)
array([[ 1.,  0.,  1.],
       [ 1.,  0.,  0.],
       [ 0.,  1.,  0.]])

有阈值

>>> binarizer = preprocessing.Binarizer(threshold=1.1)
>>> binarizer.transform(X)
array([[ 0.,  0.,  1.],
       [ 1.,  0.,  0.],
       [ 0.,  0.,  0.]])

4.标签二值化

>>> lb = preprocessing.LabelBinarizer()  
>>> lb.fit([1, 2, 6, 4, 2])  
LabelBinarizer(neg_label=0, pos_label=1)  
>>> lb.classes_  
array([1, 2, 4, 6])  
>>> lb.transform([1, 6])  
array([[1, 0, 0, 0],  
       [0, 0, 0, 1]])  

多标签显示

>>> lb.fit_transform([(1, 2), (3,)]) #(1,2)实例中就包含两个label  
array([[1, 1, 0],  
       [0, 0, 1]])  
>>> lb.classes_  
array([1, 2, 3])