利用Numpy进行鸢尾花数据集分析

Numpy进行鸢尾花数据集分析

使用鸢尾花数据集“iris_data”

鸢尾花数据集是sklearn中自带的一个经典数据集，在这个数据集中，包括了三类不同的鸢尾属植物：Iris Setosa,Iris Versicolour,Iris Virginica。每类收集50个样本，共150个样本。

数据集的特征：

• sepallength：萼片长度
• sepalwidth：萼片宽度
• petallength：花瓣长度
• petalwidth：花瓣宽度
  以上四个特征的单位都是厘米（cm）

1. 导入鸢尾花数据集，保持文本不变

import numpy as np
from sklearn.datasets import load_iris
iris=load_iris()
iris_data=np.c_[iris['data'],iris['target']]

iris_data[:10,:]#查看前十个数据

array([[5.1, 3.5, 1.4, 0.2, 0. ],
       [4.9, 3. , 1.4, 0.2, 0. ],
       [4.7, 3.2, 1.3, 0.2, 0. ],
       [4.6, 3.1, 1.5, 0.2, 0. ],
       [5. , 3.6, 1.4, 0.2, 0. ],
       [5.4, 3.9, 1.7, 0.4, 0. ],
       [4.6, 3.4, 1.4, 0.3, 0. ],
       [5. , 3.4, 1.5, 0.2, 0. ],
       [4.4, 2.9, 1.4, 0.2, 0. ],
       [4.9, 3.1, 1.5, 0.1, 0. ]])

target_map_dict={0:'Iris-setosa',1:'Iris-versicolor',2:'Iris-virginica'}

def target_map(x,map_dict):
    return map_dict[x] if x in map_dict else x
target_map_vector=np.vectorize(target_map)
iris_target_maped=target_map_vector(iris['target'],target_map_dict)

array(['Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-setosa', 'Iris-setosa',
       'Iris-setosa', 'Iris-setosa', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-versicolor', 'Iris-versicolor', 'Iris-versicolor',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica', 'Iris-virginica',
       'Iris-virginica', 'Iris-virginica'], dtype='<U15')

iris_data_map=np.c_[iris['data'],iris_target_maped]#合并后会统一数据类型

iris_data_map[0:10]

array([['5.1', '3.5', '1.4', '0.2', 'Iris-setosa'],
       ['4.9', '3.0', '1.4', '0.2', 'Iris-setosa'],
       ['4.7', '3.2', '1.3', '0.2', 'Iris-setosa'],
       ['4.6', '3.1', '1.5', '0.2', 'Iris-setosa'],
       ['5.0', '3.6', '1.4', '0.2', 'Iris-setosa'],
       ['5.4', '3.9', '1.7', '0.4', 'Iris-setosa'],
       ['4.6', '3.4', '1.4', '0.3', 'Iris-setosa'],
       ['5.0', '3.4', '1.5', '0.2', 'Iris-setosa'],
       ['4.4', '2.9', '1.4', '0.2', 'Iris-setosa'],
       ['4.9', '3.1', '1.5', '0.1', 'Iris-setosa']], dtype='<U32')

np.savetxt('iris.data',iris_data_map,fmt='%s',delimiter=',')

iris_data = np.loadtxt('iris.data', dtype=object, delimiter=',', skiprows=1)
print(iris_data[0:10])

[['4.9' '3.0' '1.4' '0.2' 'Iris-setosa']
 ['4.7' '3.2' '1.3' '0.2' 'Iris-setosa']
 ['4.6' '3.1' '1.5' '0.2' 'Iris-setosa']
 ['5.0' '3.6' '1.4' '0.2' 'Iris-setosa']
 ['5.4' '3.9' '1.7' '0.4' 'Iris-setosa']
 ['4.6' '3.4' '1.4' '0.3' 'Iris-setosa']
 ['5.0' '3.4' '1.5' '0.2' 'Iris-setosa']
 ['4.4' '2.9' '1.4' '0.2' 'Iris-setosa']
 ['4.9' '3.1' '1.5' '0.1' 'Iris-setosa']
 ['5.4' '3.7' '1.5' '0.2' 'Iris-setosa']]

2求出鸢尾属植物萼片的平均值，中位数和标准差（第一列，sepallenth）

outfile='iris.data'
sepallength = np.loadtxt(outfile,dtype=float,delimiter=',',usecols=[0])
print(sepallength[0:10])

[5.1 4.9 4.7 4.6 5.  5.4 4.6 5.  4.4 4.9]

print(np.mean(sepallength))

5.843333333333334

print(np.median(sepallength))

5.8

print(np.std(sepallength))

0.8253012917851409

3.创建一种标准化形式的鸢尾属植物萼片长度,其值正好介于0和1之间,这样最小值为0,最大值为1(第1列,sepallength)。

#方法一
aMax = np.amax(sepallength)
aMin = np.amin(sepallength)
x = (sepallength-aMin)/(aMax - aMin)
print(x[0:10])

#方法二
x = (sepallength-aMin)/np.ptp(sepallength)
print(x[:10])

[0.22222222 0.16666667 0.11111111 0.08333333 0.19444444 0.30555556
 0.08333333 0.19444444 0.02777778 0.16666667]
[0.22222222 0.16666667 0.11111111 0.08333333 0.19444444 0.30555556
 0.08333333 0.19444444 0.02777778 0.16666667]

4. 找到鸢尾属植物萼片长度的第5和第95百分位数(第1列,sepallength)。

print(np.percentile(sepallength,[5,95]))

[4.6   7.255]

5. 把iris_data数据集中的20个随机位置修改为np.nan值。

i, j = iris_data.shape
np.random.seed(20200621)
iris_data[np.random.randint(i, size=20), np.random.randint(j, size=20)] = np.nan
print(iris_data[0:10])

[['4.9' '3.0' '1.4' '0.2' 'Iris-setosa']
 ['4.7' '3.2' '1.3' '0.2' 'Iris-setosa']
 ['4.6' '3.1' '1.5' '0.2' 'Iris-setosa']
 ['5.0' '3.6' '1.4' '0.2' 'Iris-setosa']
 ['5.4' '3.9' '1.7' '0.4' 'Iris-setosa']
 ['4.6' nan '1.4' '0.3' 'Iris-setosa']
 ['5.0' '3.4' '1.5' '0.2' 'Iris-setosa']
 ['4.4' '2.9' '1.4' '0.2' 'Iris-setosa']
 ['4.9' '3.1' '1.5' '0.1' nan]
 ['5.4' '3.7' '1.5' '0.2' 'Iris-setosa']]

i, j = iris_data.shape
np.random.seed(20200620)
iris_data[np.random.choice(i, size=20), np.random.choice(j, size=20)] = np.nan
print(iris_data[0:10])

[['4.9' '3.0' '1.4' '0.2' 'Iris-setosa']
 ['4.7' '3.2' '1.3' '0.2' 'Iris-setosa']
 ['4.6' '3.1' '1.5' '0.2' 'Iris-setosa']
 ['5.0' '3.6' '1.4' '0.2' 'Iris-setosa']
 ['5.4' '3.9' '1.7' nan 'Iris-setosa']
 ['4.6' nan '1.4' '0.3' 'Iris-setosa']
 ['5.0' '3.4' '1.5' '0.2' 'Iris-setosa']
 ['4.4' '2.9' '1.4' '0.2' 'Iris-setosa']
 ['4.9' '3.1' '1.5' '0.1' nan]
 [nan '3.7' '1.5' '0.2' 'Iris-setosa']]

6. 在iris_data的sepallength中查找缺失值的个数和位置(第1列)。

outfile = r'iris.data'
iris_data = np.loadtxt(outfile, dtype=float, delimiter=',', skiprows=1, usecols=[0, 1, 2,3])                                                                           
i, j = iris_data.shape
np.random.seed(20200621)
iris_data[np.random.randint(i, size=20), np.random.randint(j, size=20)] = np.nan
sepallength = iris_data[:, 0]
x = np.isnan(sepallength)
print(sum(x)) # 6
print(np.where(x))#返回空值的位置
# (array([ 26, 44, 55, 63, 90, 115], dtype=int64),)

6
(array([ 26,  44,  55,  63,  90, 115]),)

7. 筛选具有 sepallength(第1列)< 5.0 并且 petallength(第3列)> 1.5 的 iris_data行。

outfile = 'iris.data'
iris_data = np.loadtxt(outfile,dtype=float,delimiter=',',usecols=[0,1,2,3])
sepallength = iris_data[:,0]
petallength = iris_data[:,2]
index = np.where(np.logical_and(petallength > 1.5,sepallength < 5.0))
print(index)
print(iris_data[index])

(array([ 11,  24,  29,  30,  57, 106]),)
[[4.8 3.4 1.6 0.2]
 [4.8 3.4 1.9 0.2]
 [4.7 3.2 1.6 0.2]
 [4.8 3.1 1.6 0.2]
 [4.9 2.4 3.3 1. ]
 [4.9 2.5 4.5 1.7]]

8.选择没有任何nan值的iris_data行

outfile = 'iris.data'
iris_data = np.loadtxt(outfile,dtype=float,delimiter=',',usecols=[0,1,2,3])
i,j = iris_data.shape
np.random.seed(20201201)
iris_data[np.random.randint(i,size=20),np.random.randint(j,size=20)]=np.nan
x = iris_data[np.sum(np.isnan(iris_data),axis=1)==0]
#print(x)
print(x[0:10])

[[5.1 3.5 1.4 0.2]
 [4.9 3.  1.4 0.2]
 [4.7 3.2 1.3 0.2]
 [5.  3.6 1.4 0.2]
 [5.4 3.9 1.7 0.4]
 [4.6 3.4 1.4 0.3]
 [5.  3.4 1.5 0.2]
 [4.4 2.9 1.4 0.2]
 [4.9 3.1 1.5 0.1]
 [5.4 3.7 1.5 0.2]]

计算iris_data中sepallengrh（第一列）和petallength（第三列）之间的相关系数

outfile = 'iris.data'
iris_data = np.loadtxt(outfile,dtype=float,delimiter=',',usecols=[0,1,2,3])
sepalLength = iris_data[:, 0]
petalLength = iris_data[:, 2]
#方法一
m1=np.mean(sepalLength)
m2=np.mean(petalLength)
cov = np.dot(sepalLength - m1 ,petalLength - m2)
std1 = np.sqrt(np.dot(sepalLength - m1,sepalLength - m1))
std2 = np.sqrt(np.dot(petalLength - m2,petalLength - m2))
print(cov/(std1*std2))
#方法二
x = np.mean((sepalLength - m1)*(petalLength - m2))
y = np.std(sepalLength)*np.std(petalLength)
print(x/y)
#方法三
x = np.cov(sepalLength,petalLength,ddof=False)
y = np.std(sepalLength)*np.std(petalLength)
print(x[0,1]/y)
#方法四
x = np.corrcoef(sepalLength,petalLength)
print(x)

0.8717537758865833
0.8717537758865831
0.8717537758865835
[[1.         0.87175378]
 [0.87175378 1.        ]]

10.找出iris_data是否有任何缺失值

x = np.isnan(iris_data)
print(np.any(x))

False

11. 在numpy数组中将所有出现的nan替换为0。

outfile = 'iris.data'
iris_data = np.loadtxt(outfile,dtype=float,delimiter=',',usecols=[0,1,2,3])
i,j = iris_data.shape
np.random.seed(20201201)
iris_data[np.random.randint(i,size=20),np.random.randint(j,size=20)] = np.nan
iris_data[np.isnan(iris_data)] = 0
print(iris_data[0:10])

[[5.1 3.5 1.4 0.2]
 [4.9 3.  1.4 0.2]
 [4.7 3.2 1.3 0.2]
 [4.6 0.  1.5 0.2]
 [5.  3.6 1.4 0.2]
 [5.4 3.9 1.7 0.4]
 [4.6 3.4 1.4 0.3]
 [5.  3.4 1.5 0.2]
 [4.4 2.9 1.4 0.2]
 [4.9 3.1 1.5 0.1]]

12. 找出鸢尾属植物物种中的唯一值和唯一值出现的数量。

outfile = 'iris.data'
iris_data = np.loadtxt(outfile,dtype=object,delimiter=',',usecols=[4])
x = np.unique(iris_data,return_counts=True)
print(x)

(array(['Iris-setosa', 'Iris-versicolor', 'Iris-virginica'], dtype=object), array([50, 50, 50]))

13. 将 iris_data 的花瓣长度(第3列)以形成分类变量的形式显示。定义:Less than 3 -->‘small’;3-5 --> ‘medium’;’>=5 --> ‘large’。

outfile = 'iris.data'
iris_data = np.loadtxt(outfile,dtype=float,delimiter=',',usecols=[0,1,2,3])
petal_length_bin = np.digitize(iris_data[:,2],[0,3,5,10])
label_map = {1:'small',2:'median',3:'large',4:np.nan}
petal_length_cat = [label_map[x] for x in petal_length_bin]
print(petal_length_cat[0:10])

['small', 'small', 'small', 'small', 'small', 'small', 'small', 'small', 'small', 'small']

petal_length_bin

array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 2, 2, 2, 2, 2, 3, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 2, 3, 3, 3,
       3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 3, 2, 3, 3, 2, 2, 3, 3, 3, 3,
       3, 3, 3, 3, 3, 3, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3])

14. 在 iris_data 中创建一个新列,其中 volume 是 (pi x petallength x sepallength ^ 2)/ 3 。

outfile = 'iris.data'
iris_data = np.loadtxt(outfile,dtype=object,delimiter=',')
sepalLength = iris_data[:,0].astype(float)
petalLength = iris_data[:,2].astype(float)
volume = (np.pi *petalLength*sepalLength**2 )/3
volume=volume[:,np.newaxis]
iris_data = np.concatenate([iris_data,volume],axis=1)
print(iris_data[0:10])

[['5.1' '3.5' '1.4' '0.2' 'Iris-setosa' 38.13265162927291]
 ['4.9' '3.0' '1.4' '0.2' 'Iris-setosa' 35.200498485922445]
 ['4.7' '3.2' '1.3' '0.2' 'Iris-setosa' 30.0723720777127]
 ['4.6' '3.1' '1.5' '0.2' 'Iris-setosa' 33.238050274980004]
 ['5.0' '3.6' '1.4' '0.2' 'Iris-setosa' 36.65191429188092]
 ['5.4' '3.9' '1.7' '0.4' 'Iris-setosa' 51.911677007917746]
 ['4.6' '3.4' '1.4' '0.3' 'Iris-setosa' 31.022180256648003]
 ['5.0' '3.4' '1.5' '0.2' 'Iris-setosa' 39.269908169872416]
 ['4.4' '2.9' '1.4' '0.2' 'Iris-setosa' 28.38324242763259]
 ['4.9' '3.1' '1.5' '0.1' 'Iris-setosa' 37.714819806345474]]

15. 随机抽鸢尾属植物的种类,使得Iris-setosa的数量是Iris-versicolor和Iris-virginica数量的两倍。

species = np.array(['Iris-setosa','Iris-versicolor','Iris-virginica'])
species_out = np.random.choice(species,1000,p=[0.5,0.25,0.25])
print(np.unique(species_out,return_counts=True))

(array(['Iris-setosa', 'Iris-versicolor', 'Iris-virginica'], dtype='<U15'), array([508, 256, 236]))

16. 根据 sepallength 列对数据集进行排序。

outfilr ='iris.data'
iris_data = np.loadtxt(outfile,dtype=object,delimiter=',')
sepalLength = iris_data[:,0]
index = np.argsort(sepalLength)
print(iris_data[index][0:10])

[['4.3' '3.0' '1.1' '0.1' 'Iris-setosa']
 ['4.4' '3.2' '1.3' '0.2' 'Iris-setosa']
 ['4.4' '3.0' '1.3' '0.2' 'Iris-setosa']
 ['4.4' '2.9' '1.4' '0.2' 'Iris-setosa']
 ['4.5' '2.3' '1.3' '0.3' 'Iris-setosa']
 ['4.6' '3.6' '1.0' '0.2' 'Iris-setosa']
 ['4.6' '3.1' '1.5' '0.2' 'Iris-setosa']
 ['4.6' '3.4' '1.4' '0.3' 'Iris-setosa']
 ['4.6' '3.2' '1.4' '0.2' 'Iris-setosa']
 ['4.7' '3.2' '1.3' '0.2' 'Iris-setosa']]

17. 在鸢尾属植物数据集中找到最常见的花瓣长度值(第3列)。

outfile = 'iris.data'
iris_data=np.loadtxt(outfile,dtype=object,delimiter=',')
petalLength = iris_data[:,2]
vals,counts=np.unique(petalLength,return_counts=True)
print(vals[np.argmax(counts)])
print(np.amax(counts))

1.4
13

18. 在鸢尾花数据集的 petalwidth(第4列)中查找第一次出现的值大于1.0的位置。

outfile ='iris.data'
iris_data = np.loadtxt(outfile,dtype=float,delimiter=',',usecols=[0,1,2,3])
petalWidth = iris_data[:,3]
index = np.where(petalWidth > 1.0)
print(index)
print(index[0][0])

(array([ 50,  51,  52,  53,  54,  55,  56,  58,  59,  61,  63,  64,  65,
        66,  68,  69,  70,  71,  72,  73,  74,  75,  76,  77,  78,  80,
        82,  83,  84,  85,  86,  87,  88,  89,  90,  91,  92,  94,  95,
        96,  97,  98,  99, 100, 101, 102, 103, 104, 105, 106, 107, 108,
       109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121,
       122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134,
       135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147,
       148, 149]),)
50