sklearn加载外部数据集

1.使用numpy.loadtxt

2.解决Arff格式的方案

1. 参考.arff files with scikit-learn? & LIAC-ARFF v2.1
2. 使用scipy.io.arff.loadarff

from scipy.io import arff

dataset=arff.loadarff("D:/res/weather.nominal.arff")
print(dataset)

In [24]: type(dataset)
Out[24]: tuple
In [25]: type(dataset[0])
Out[25]: numpy.ndarray
In [26]: type(dataset[1])
Out[26]: scipy.io.arff.arffread.MetaData

In [40]: dataset[0]
Out[40]:
array([(b'sunny', b'hot', b'high', b'FALSE', b'no'),
       (b'sunny', b'hot', b'high', b'TRUE', b'no'),
       (b'overcast', b'hot', b'high', b'FALSE', b'yes'),
       (b'rainy', b'mild', b'high', b'FALSE', b'yes'),
       (b'rainy', b'cool', b'normal', b'FALSE', b'yes'),
       (b'rainy', b'cool', b'normal', b'TRUE', b'no'),
       (b'overcast', b'cool', b'normal', b'TRUE', b'yes'),
       (b'sunny', b'mild', b'high', b'FALSE', b'no'),
       (b'sunny', b'cool', b'normal', b'FALSE', b'yes'),
       (b'rainy', b'mild', b'normal', b'FALSE', b'yes'),
       (b'sunny', b'mild', b'normal', b'TRUE', b'yes'),
       (b'overcast', b'mild', b'high', b'TRUE', b'yes'),
       (b'overcast', b'hot', b'normal', b'FALSE', b'yes'),
       (b'rainy', b'mild', b'high', b'TRUE', b'no')],
      dtype=[('outlook', 'S8'), ('temperature', 'S4'), ('humidity', 'S6'), ('win
dy', 'S5'), ('play', 'S3')])

In [41]: dataset[1]
Out[41]:
Dataset: weather.symbolic
        outlook's type is nominal, range is ('sunny', 'overcast', 'rainy')
        temperature's type is nominal, range is ('hot', 'mild', 'cool')
        humidity's type is nominal, range is ('high', 'normal')
        windy's type is nominal, range is ('TRUE', 'FALSE')
        play's type is nominal, range is ('yes', 'no')

先来了解什么是dtype类型
数组元素(numpy.array())的类型通过dtype属性获得。

In [43]: import numpy as np

In [44]: a=np.array([1,2,3,4,5])

In [45]: a
Out[45]: array([1, 2, 3, 4, 5])

In [46]: a.dtype
Out[46]: dtype('int32')

In [47]: b=np.array([1,2,3,4,5],dtype=np.float)

In [48]: b
Out[48]: array([1., 2., 3., 4., 5.])

In [49]: b.dtype
Out[49]: dtype('float64')

而且，每一种数据类型都有几种字符串表达形式，我们可以使用typeDict字典来查询某种字符串所代表的数据类型，比如“d”和“double”都是float64数据类型：

In [51]: np.typeDict['d']
Out[51]: numpy.float64

In [52]: np.typeDict['double']
Out[52]: numpy.float64

In [53]: np.typeDict['f']
Out[53]: numpy.float32

In [54]: np.typeDict['I']
Out[54]: numpy.uint32

In [57]: dt=np.dtype([('name','S8'),('age','I'),('grade','f')])

In [58]: student=np.array([('lili',14,90),('dada',13,67),('jiji',14,87)],dtype=
    ...: dt)

In [59]: student
Out[59]:
array([(b'lili', 14, 90.), (b'dada', 13, 67.), (b'jiji', 14, 87.)],
      dtype=[('name', 'S8'), ('age', '<u4'), ('grade', '<f4')])

In [60]: student['name']
Out[60]: array([b'lili', b'dada', b'jiji'], dtype='|S8')

In [61]: student['age']
Out[61]: array([14, 13, 14], dtype=uint32)

In [62]: student[0]
Out[62]: (b'lili', 14, 90.)