pandas的使用
pandas中的解析函数:
函数名 | 说明 |
---|---|
read_csv() | 默认分隔符为逗号 |
read_table() | 默认分隔符为制表符(‘\t’) |
read_fwf() | 读取定宽列格式数据,也就是说没有分隔符 |
read_clipboard() | 读取剪切板中的数据.在将网页转换为表格时很有用 |
在创建时,我们可以指定一下的属性:
- 路径
path
- 分隔符
sep
- 列名
header OR name
- 索引
index_col
- 跳行
skiprows
- 缺失值
na_values
- 行数
nrows
- 文件块
chunksize
还有很多,诸如comment, parse_dates, keep_date_col, converters, dayfirst, date_parser, iterator, skip_footer, verbose, encoding, squeeze, thousands
举例子
函数示范举例:
import pandas as pd
- 分隔符1
>>> pd.read_table('ex3.txt',sep='\s+')
A B C
aaa -0.264438 -1.026059 -0.619500
bbb 0.927272 0.302904 -0.032399
ccc -0.264273 -0.386314 -0.217601
ddd -0.871858 -0.348382 1.100491
- 列名
列名为None:
>>> pd.read_csv('ex2.csv',header=None)
0 1 2 3 4
0 1 2 3 4 hello
1 5 6 7 8 world
2 9 10 11 12 foo
自定义列名:
>>> pd.read_csv('ex2.csv', names=['a','b','c','d','message'])
a b c d message
0 1 2 3 4 hello
1 5 6 7 8 world
2 9 10 11 12 foo
- 索引
注:如果希望列名中的某一个列做成DataFrame的索引,通过index_col参数指定.
>>> pd.read_csv('ex2.csv', names=['a','b','c','d','message'], index_col='message')
a b c d
message
hello 1 2 3 4
world 5 6 7 8
foo 9 10 11 12
当然我们也可以做一个层次化索引.
>>> pd.read_csv('csv_mindex.csv')
key1 key2 value1 value2
0 one a 1 2
1 one b 3 4
2 one c 5 6
3 one d 7 8
4 two a 9 10
5 two b 11 12
6 two c 13 14
7 two d 15 16
>>> pd.read_csv('csv_mindex.csv', index_col=['key1', 'key2'])
value1 value2
key1 key2
one a 1 2
b 3 4
c 5 6
d 7 8
two a 9 10
b 11 12
c 13 14
d 15 16
- 跳行2
>>> pd.read_csv('ex4.csv', skiprows=[0,2,3])
a b c d message
0 1 2 3 4 hello
1 5 6 7 8 world
2 9 10 11 12 foo
- 缺失值
>>> res = pd.read_csv('ex5.csv')
>>> res
something a b c d message
0 one 1 2 3 4 NaN
1 two 5 6 NaN 8 world
2 three 9 10 11 12 foo
>>> pd.isnull(res)
something a b c d message
0 False False False False False True
1 False False False True False False
2 False False False False False False
>>> pd.read_csv('ex5.csv', na_values=['NULL'])
something a b c d message
0 one 1 2 3 4 NaN
1 two 5 6 NaN 8 world
2 three 9 10 11 12 foo
用字典为各列指定NA值
#可以看到[0][c],[2][message]变为NA值
>>> dict={'c':[1000,3],'message':['NA','foo']}
>>> pd.read_csv('ex5.csv', na_values=dict)
something a b c d message
0 one 1 2 NaN 4 NaN
1 two 5 6 NaN 8 world
2 three 9 10 11 12 NaN
- 行数
>>> pd.read_csv('ex6.csv', nrows=10)
one two three four key
0 0.467976 -0.038649 -0.295344 -1.824726 L
1 -0.358893 1.404453 0.704965 -0.200638 B
2 -0.501840 0.659254 -0.421691 -0.057688 G
3 0.204886 1.074134 1.388361 -0.982404 R
4 0.354628 -0.133116 0.283763 -0.837063 Q
5 1.817480 0.742273 0.419395 -2.251035 Q
6 -0.776764 0.935518 -0.332872 -1.875641 U
7 -0.913135 1.530624 -0.572657 0.477252 K
8 0.358480 -0.497572 -0.367016 0.507702 S
9 -1.740877 -1.160417 -1.637830 2.172201 G
- 文件块
>>> chunk = pd.read_csv('ex6.csv', chunksize = 1000)
>>> from pandas import Series
>>> tot = Series([])#Series是一种类似于数组的对象,它由一组数据以及一组与之相关的数据标签(即索引)组成
>>> for piece in chunk:
#value_counts()统计这个块中相同值的个数
#块中缺失值几即赋值为0
... tot = tot.add(piece['key'].value_counts(), fill_value=0)
...
>>> tot = tot.sort_values(ascending=False)
>>> tot[:10]
E 368
X 364
L 346
O 343
Q 340
M 338
J 337
F 335
K 334
H 330
dtype: float64