我正在使用多处理.池在
每个进程都要查询mysql数据库。在
我当前连接到数据库一次,然后共享进程之间的连接
它很管用,但偶尔我会犯一些奇怪的错误。我已经确认错误是在查询数据库时引起的。在
我想问题是因为所有进程都使用相同的连接。在这对吗?
所以我查了一下班级pooling.MySQLConnectionPool在
如果我能理解的话。我将设置一个具有多个连接的池,并在进程之间共享该池。然后,每个进程都会查看该池,如果有可用的连接,请使用该池,否则请等待释放连接。在这对吗?
“mata”似乎首先证实了我的怀疑,但同时他否认了在进程之间建立共享池的用法sharing a database connection (or connection pool) between different processes would be a bad idea (and i highly doubt it would even work correctly),
相反,他建议so each process using it's own connections is actually what you should aim for.
那是什么意思?在我应该为每个工作线程创建一个连接吗?
那么mysql池有什么用呢?
mata在他的回答中给出的例子似乎足够合理,但我不理解整个池的传递是init参数p = Pool(initializer=init)为什么?(正如ph嫒singer在评论中指出的那样,这不是一个好的解决方案)
更换挡块池.map()方法Pool.map_异步()并从池发送到映射异步(q,ConnObj)的连接就足够了?在这对吗?
在评论中提到The only way of utilizing one single pool with many processes is having one dedicated process which does all the db access communicate with it using a queueIf you need large numbers of concurrent workers, but they're not using the DB all the time, you should have a group of database worker processes that handle all database access and exchange data with your other worker processes. Each database worker process has a DB connection. The other processes only talk to the database via your database workers.
Python's multiprocessing queues, fifos, etc offer appropriate messaging features for that.这真的正确吗?
mysql池的目的不是处理进程的请求并将它们转发到可用的连接吗?在
现在我很困惑。。。在