科研最后一步之Pandas基础百问篇

系列文章目录

第一章 科研最后一步之Pandas基础百问篇

---

---

Pandas

Pandas是一个强大的分析结构化数据的工具集，基于NumPy构建，提供了高级数据结构和数据操作工具，它是使Python成为强大而高效的数据分析环境的重要因素之一。Pandas 广泛应用在学术、金融、统计学等各个数据分析领域。
本章内容以问题为导向，方便用户快速获取相关功能，下面案例可供参考。

---

1. 如何导入pandas并查看其版本？

import numpy as np  # optional
import pandas as pd
print(pd.__version__)
print(pd.show_versions(as_json=True))

1.2.4
>{'system': {'commit': '2cb96529396d93b46abab7bbc73a208e708c642e', 'python': '3.8.8.final.0', 'python-bits': 64, 'OS': 'Windows', 'OS-release': '10', 'Version': '10.0.19041', 'machine': '', 'processor': '', 'byteorder': 'little', 'LC_ALL': None, 'LANG': None, 'LOCALE': {'language-code': 'Chinese (Simplified)_China', 'encoding': '936'}}, 'dependencies': {'pandas': '1.2.4', 'numpy': '1.20.1', 'pytz': '2021.1', 'dateutil': '2.8.1', 'pip': '21.0.1', 'setuptools': '52.0.0.post20210125', 'Cython': '0.29.23', 'pytest': '6.2.3', 'hypothesis': None, 'sphinx': '4.0.1', 'blosc': None, 'feather': None, 'xlsxwriter': '1.3.8', 'lxml.etree': '4.6.3', 'html5lib': '1.1', 'pymysql': None, 'psycopg2': None, 'jinja2': '2.11.3', 'IPython': '7.22.0', 'pandas_datareader': None, 'bs4': '4.9.3', 'bottleneck': '1.3.2', 'fsspec': '0.9.0', 'fastparquet': None, 'gcsfs': None, 'matplotlib': '3.3.4', 'numexpr': '2.7.3', 'odfpy': None, 'openpyxl': '3.0.7', 'pandas_gbq': None, 'pyarrow': '7.0.0', 'pyxlsb': None, 's3fs': None, 'scipy': '1.6.2', 'sqlalchemy': '1.4.7', 'tables': '3.6.1', 'tabulate': None, 'xarray': None, 'xlrd': '2.0.1', 'xlwt': '1.3.0', 'numba': '0.53.1'}}
None

2. 如何从list列表、numpy数组和dict字典创建series序列？

# Inputs
import numpy as np
mylist = list('abcedfghijklmnopqrstuvwxyz')
myarr = np.arange(26)
mydict = dict(zip(mylist, myarr))

# Solution
ser1 = pd.Series(mylist)
ser2 = pd.Series(myarr)
ser3 = pd.Series(mydict)
print(ser3.head())

a    0
b    1
c    2
e    3
d    4
dtype: int64

3.如何将series序列转化为DataFrame数据框？

# Input
mylist = list('abcedfghijklmnopqrstuvwxyz')
myarr = np.arange(26)
mydict = dict(zip(mylist, myarr))
ser = pd.Series(mydict)

# Solution
df = ser.to_frame().reset_index()
print(df.head())

  index  0
0     a  0
1     b  1
2     c  2
3     d  4
4     e  3

4.如何将多个Series序列合并为一个DataFrame数据框？

# Input
import numpy as np
ser1 = pd.Series(list('abcedfghijklmnopqrstuvwxyz'))
ser2 = pd.Series(np.arange(26))

# Solution 1
df = pd.concat([ser1, ser2], axis=1)

# Solution 2
df = pd.DataFrame({'col1': ser1, 'col2': ser2})
print(df.head())

  col1  col2
0    a     0
1    b     1
2    c     2
3    e     3
4    d     4

5.如何修改series序列index索引名称？

# Input
ser = pd.Series(list('abcedfghijklmnopqrstuvwxyz'))

# Solution
ser.name = 'alphabets'
ser.head()

0    a
1    b
2    c
3    e
4    d
Name: alphabets, dtype: object

6. 如何获取序列seriesA不存在seriesB中元素？

# Input
ser1 = pd.Series([1, 2, 3, 4, 5])
ser2 = pd.Series([4, 5, 6, 7, 8])

# Solution
ser1[~ser1.isin(ser2)]

0    1
1    2
2    3
dtype: int64

7. 如何获得序列seriesA和seriesB差集？

# Input
ser1 = pd.Series([1, 2, 3, 4, 5])
ser2 = pd.Series([4, 5, 6, 7, 8])

# Solution
ser_u = pd.Series(np.union1d(ser1, ser2))  # union
ser_i = pd.Series(np.intersect1d(ser1, ser2))  # intersect
ser_u[~ser_u.isin(ser_i)]

0    1
1    2
2    3
5    6
6    7
7    8
dtype: int64

8. 如何获得series序列的最小值、第25百分位数、中位数、75位数和最大值？

# Input
state = np.random.RandomState(100)
ser = pd.Series(state.normal(10, 5, 25))

# Solution
np.percentile(ser, q=[0, 25, 50, 75, 100])

array([  1.39267584,   6.49135133,  10.2578186 ,  13.06985067,  25.80920994])

9.如何统计series序列中元素的频数?

# Input
ser = pd.Series(np.take(list('abcdefgh'), np.random.randint(8, size=30)))

# Solution
ser.value_counts()

f    8
g    7
b    6
c    4
a    2
e    2
h    1
dtype: int64

10. 如何只保留Series序列中频数前两位，其它的值替换为other？

# Input
np.random.RandomState(100)
ser = pd.Series(np.random.randint(1, 5, [12]))

# Solution
print("Top 2 Freq:", ser.value_counts())
ser[~ser.isin(ser.value_counts().index[:2])] = 'Other'
ser

Top 2 Freq: 1    5
4    3
2    2
3    2
dtype: int64

0     Other
1         1
2         1
3         1
4         4
5     Other
6         4
7     Other
8         1
9         1
10        4
11    Other
dtype: object
dtype: object

11. 如何将Series序列均分为10个区间、每个值使用区间名称标记

# Input
ser = pd.Series(np.random.random(20))
print(ser.head())

# Solution
pd.qcut(ser, q=[0, .10, .20, .3, .4, .5, .6, .7, .8, .9, 1], 
        labels=['1st', '2nd', '3rd', '4th', '5th', '6th', '7th', '8th', '9th', '10th']).head()

0    0.763815
1    0.522430
2    0.357147
3    0.138867
4    0.317274
dtype: float64
0    8th
1    4th
2    3rd
3    2nd
4    3rd
dtype: category
Categories (10, object): ['1st' < '2nd' < '3rd' < '4th' ... '7th' < '8th' < '9th' < '10th']

12. 如何将一维numpy数组转换为二维数据框？

# Input
ser = pd.Series(np.random.randint(1, 10, 35))

# Solution
df = pd.DataFrame(ser.values.reshape(7,5))
print(df)

   0  1  2  3  4
0  1  2  1  2  5
1  1  2  4  5  2
2  1  3  3  2  8
3  8  6  4  9  6
4  2  1  1  8  5
5  3  2  8  5  6
6  1  5  5  4  6

13. 如何取出Series序列中满足条件数据的位置索引index？

np.random.seed(999) 
ser = pd.Series(np.random.randint(1, 10, 7))
print(ser)
np.argwhere(ser.values % 3 == 0) #查找3的倍数

0    1
1    6
2    2
3    9
4    2
5    4
6    1
dtype: int32
array([[1],
       [3]], dtype=int64)

14. 如何从Series序列中提取指定位置的元素

# Input
ser = pd.Series(list('abcdefghijklmnopqrstuvwxyz'))
pos = [0, 4, 8, 14, 20]

# Solution
ser.take(pos)

0     a
4     e
8     i
14    o
20    u
dtype: object

15. 如何将Series序列水平、垂直合并？

# Input
ser1 = pd.Series(range(5))
ser2 = pd.Series(list('abcde'))

# Output
# Vertical
ser1.append(ser2)

# Horizontal
df = pd.concat([ser1, ser2], axis=1)
print(df)

# Vertical
0    0
1    1
2    2
3    3
4    4
0    a
1    b
2    c
3    d
4    e
dtype: object
# Horizontal
   0  1
0  0  a
1  1  b
2  2  c
3  3  d
4  4  e

---

未完待更