强化学习中各式采样方法

强化学习中，在收集了不少数据之后，要从这批数据中抽取一些样本进行强化学习训练，这里主要针对了可能的np array，list和deque，用random 和np自带的函数进行5种已存在的采样方式进行了测试。

"""
Created on Thu Apr 16 21:50:11 2020

@author: C. Z.
"""

"""
测试一下各式采样，主要包括了
choices(population, weights=None, *, cum_weights=None, k=1) method of random.Random instance
Return a k sized list of population elements chosen with replacement.
——random.choices()


Choose a random element from a non-empty sequence.
——random.choice() 


sample(population, k) method of random.Random instance
Chooses k unique random elements from a population sequence or set.
    
Returns a new list containing elements from the population while
leaving the original population unchanged.
——random.sample()


This is an alias of `random_sample`. See `random_sample`  for the completedocumentation.
——np.random.sample() 


Generates a random sample from a given 1-D array
——np.random.choice() 
"""

这里测试了一种最常见的一种数据样本，每条数据既有数字，又有逻辑，字符和列表，废话不多说，下面是测试代码和结果
（注：纯数字构成的np array的数据采样，推荐np 自带的sample和choice满足各种需求。）

import random
import numpy as np
from collections import deque


a = [[1,2,3,4,[5,6,7,8,],True,'a'],
     [1,2,3,4,[5,6,7,8,],False,'b'],
     [1,2,3,4,[5,6,7,8,],False,'c'],
     [1,2,3,4,[5,6,7,8,],True,'d'],
     [1,2,3,4,[5,6,7,8,],True,'e'],
     [1,2,3,4,[5,6,7,8,],True,'f'],
     [1,2,3,4,[5,6,7,8,],True,'g'],
     [1,2,3,4,[5,6,7,8,],True,'h'],
     [1,2,3,4,[5,6,7,8,],True,'i'],
     [1,2,3,4,[5,6,7,8,],True,'j'],
     [1,2,3,4,[5,6,7,8,],True,'k'],
     ]

b =deque(a)

c = np.array(a)

random.choice

random.choice(a)

random.choice 函数只可以随机选取一行

[1, 2, 3, 4, [5, 6, 7, 8], True, 'h']

random.choices

random.choices(a,k=15)
random.choices(b,k=15)

random.choices 函数只可以随机选取多行，最重要的是它可以选取超过列本身的行数，这在某些程序写起来的时候特别方便。

[[1, 2, 3, 4, [5, 6, 7, 8], True, 'f'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'k'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'i'],
 [1, 2, 3, 4, [5, 6, 7, 8], False, 'b'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'k'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'f'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'h'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'e'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'f'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'h'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'a'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'i'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'e'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'h'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'g']]

[[1, 2, 3, 4, [5, 6, 7, 8], False, 'b'],
 [1, 2, 3, 4, [5, 6, 7, 8], False, 'c'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'j'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'k'],
 [1, 2, 3, 4, [5, 6, 7, 8], False, 'c'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'h'],
 [1, 2, 3, 4, [5, 6, 7, 8], False, 'c'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'a'],
 [1, 2, 3, 4, [5, 6, 7, 8], False, 'b'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'f'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'f'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'e'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'i'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'e'],
 [1, 2, 3, 4, [5, 6, 7, 8], True, 'g']]

random.sample

random.sample(a,k=15)

random.sample就没有上面的choices这么好用了，虽然也可以选取多行，但是不能超过样本本身的数量。我想应该是sample是不放回抽取，choice是放回抽取的缘故吧。

Sample larger than population or is negative

np.random.choice和np.random.sample

这两种采样方式都是基于random的，但是它们处理的对象都最好只用于最简单的np array数据。

np.random.choice(a)
np.random.choice(b)
np.random.choice(c)

结果：

ValueError: a must be 1-dimensional

np.random.sample(a)
np.random.sample(b)
np.random.sample(c)

结果：

TypeError: 'list' object cannot be interpreted as an integer
TypeError: 'list' object cannot be interpreted as an integer
TypeError: only integer scalar arrays can be converted to a scalar index