python使用celery 报错 ： OSError handle is closed OSError handle is closed

报错堆栈信息如下

[2019-09-30 11:19:00,756: ERROR/ForkPoolWorker-85] Thread 'ResultHandler' crashed: OSError('handle is closed',)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/billiard/pool.py", line 478, in run
    return self.body()
  File "/usr/lib/python2.7/site-packages/billiard/pool.py", line 866, in body
    self.finish_at_shutdown()
  File "/usr/lib/python2.7/site-packages/billiard/pool.py", line 883, in finish_at_shutdown
    ready, task = poll(1.0)
  File "/usr/lib/python2.7/site-packages/billiard/pool.py", line 1326, in _poll_result
    if self._outqueue._reader.poll(timeout):
  File "/usr/lib/python2.7/site-packages/billiard/connection.py", line 285, in poll
    self._check_closed()
  File "/usr/lib/python2.7/site-packages/billiard/connection.py", line 165, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed
[2019-09-30 11:19:01,067: ERROR/MainProcess] Process 'ForkPoolWorker-85' pid:10136 exited with 'exitcode 1'
[2019-09-30 11:19:11,483: ERROR/MainProcess] Task handler raised error: WorkerLostError('Worker exited prematurely: exitcode 1.',)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/billiard/pool.py", line 1223, in mark_as_worker_lost
    human_status(exitcode)),
WorkerLostError: Worker exited prematurely: exitcode 1.

定位问题

• 通过Google搜索到时因为任务中有用到gevent模块去查询数据库，导致报错

解决方案

• 通过放弃使用gevent 转而使用ThreadPoolExecutor来实现多任务解决这个问题

反思

• 一开始是找了很久的Google，并没有搜到这个报错(当然。。。还是用的是Google，如果是百度可能就找不到了)
  • 第一，搜索的关键词不对，导致刚开始没有搜到（加入了中文，导致搜索引擎会吧全英结果往后排）
  • 第二，其实报错OsEroor ‘handle is closed’ 很多时候是报错多线程执行问题，我应该通过这个想到可能是gevent包导致的
  • 第三，有一部分古董代码写的有点赶工的样子，导致着实不方便定位问题(警示自己要避免写这样的代码~~)

github关于此问题的讨论