boto线程不安全的解决办法

boto线程不安全：

https://www.coder.work/article/368131

解决办法：一个进程初始化一个boto客户端

def initializer():
    global s3_client
    config.read('config/my.cfg')
    CONS_AK = config.get('default', 'aws_access_key_id')
    CONS_SK = config.get('default', 'aws_secret_access_key')
    bucket_name = config.get('default', 'bucket_name')
    host = config.get('default', 'host')
    port = config.get('default', 'port')
    s3_client = boto.connect_s3(
              aws_access_key_id=CONS_AK,
              aws_secret_access_key=CONS_SK,
              host=str(host),
              port=int(port),
              is_secure=False,
              calling_format=boto.s3.connection.OrdinaryCallingFormat()
              )
    print('init s3_client')

 

with Pool(96, initializer, ())as p:
        p.map(gen_image, params)

参考：https://docs.python.org/zh-cn/3/library/multiprocessing.html#module-multiprocessing.pool