pandas读取csv相对路径_pandas读取csv文件并采样

当csv文件比较大的时候，往往希望在读取的时候就按照一定规则采样，免得浪费时间。

参考这篇回答：

Read a small random sample from a big CSV file into a Python data frame​stackoverflow.com

主要用两种办法，随机采样和等间隔采样。

随机采样：

import pandas as pd
import random
p = 0.01  # 1% of the lines
# keep the header, then take only 1% of lines
# if random from [0,1] interval is greater than 0.01 the row will be skipped
df = pd.read_csv(
         filename,
         header=0, 
         skiprows=lambda i: i>0 and random.random() > p
)

等间隔采样：

n = 100  # every 100th line = 1% of the lines
df = pd.read_csv(filename, header=0, skiprows=lambda i: i % n != 0)