多进程 报错 TypeError: cannot pickle ‘_thread.lock‘ object

问题

在运行flask web时候也需要运行websocket
调试websocket对象用协程，线程都无法两者都运行，因此使用多线程，多线程也有报错 TypeError: cannot pickle ‘_thread.lock’ object

代码：

class ws_utils(object):
    def __init__(self, ws_config, redis):
        self.host = ws_config['ip']
        self.port = ws_config['port']
        self.redis = redis
        self.cache = {}

if __name__ == '__main__':
	global redis
    redis = Redis(config['redis'])
    global database
    database = Database(config['manager-server'])
    # 2.register nacos
    nacos_register()
    # 3.ws task: new process
    ws = ws_utils(config['ws'], redis)
    p = Process(target=ws.runWS)
    p.start()
    server = config['server']
    app.run(host=server["ip"], port=server["port"])

在多进程执行报错

Traceback (most recent call last):
  File "C:\Users\miaochangbin\PycharmProjects\eduCrawler\main.py", line 138, in <module>
    p.start()
  File "D:\Program Files\python\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "D:\Program Files\python\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "D:\Program Files\python\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "D:\Program Files\python\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "D:\Program Files\python\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\Program Files\python\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "D:\Program Files\python\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

原因

根据晚上查找资料说是在开启进程，会将信息序列化后传递到子进程，报错就是在信息进行序列化时候。
这边信息报错传递子进程的对象信息（属性以及属性的属性）。
猜测是上面ws_utils对象进行序列化问题，比较容易猜测，redis对象序列化存在问题。
验证下(将redis对象改成None类型)：

  ws = ws_utils(config['ws'], redis)

改成

   ws = ws_utils(config['ws'], None)

再次运行后，执行正常

解决

知道上面原因就好解决
将对象的属性改成可序列化即可
可以在ws_utils对象中创建全局变量redis，而不直接传redis对象，而传redis配置（或者直接从配置文件读取）部分

import asyncio
import websockets
from threading import Thread
from common.utils.redisUtils import Redis


class ws_utils(object):
	# 1.修改参数 redis -> redis_config
    def __init__(self, ws_config, redis_config):
        self.host = ws_config['ip']
        self.port = ws_config['port']
        self.redis_config = redis_config
        self.cache = {}

    def runWS(self):
    	# 2.多进程方法中创建redis对象全局变量
        global redis
        redis = Redis(self.redis_config)
        
if __name__ == '__main__':
	global redis
    redis = Redis(config['redis'])
    global database
    database = Database(config['manager-server'])
    # register nacos
    nacos_register()
    # ws task: new process
    # 3.参数 redis配置
    ws = ws_utils(config['ws'], config['redis'])
    p = Process(target=ws.runWS)
    p.start()
    server = config['server']
    app.run(host=server["ip"], port=server["port"])

运行规后达到效果，flask能运行，ws也能访问

参考

https://blog.csdn.net/LutingWang/article/details/124203032