pyqt5 多进程,在PyQt5应用程序中，可以并行运行sklearn而不冻结

Is it possible to run, in a qt application, without freezing the gui, let's say a sklearn gird search that use several jobs parallel (n_jobs > 1)? The problem is that joblib that is used for parallelizing sklearn code cannot run multiprocess into a thread.

For example, I'm using Gridsearch to find the best parameters for a svr, which is quite computionnaly intensive.

This question has been asked several times, but no solution found:

multiprocessing-backed-parallel-loops-cannot-be-nested-below-threads,the threading.current_thread().name = 'MainThread' workaround does not work after the issue has been fixed

joblib-parallel-uses-only-one-core-if-started-from-qthread, rewrite the task using multiprocessing.Pool(processes=4). This method is not applicable for gridsearch embed njobs.

And any insight why this is purposely not supported (it a feature) ? It seems like it something that would be quite useful ?

解决方案

From my understanding of the issue, the problem resides with the default backend used by joblib, namely loky.

After some digging through the joblib and sklearn documentation, I resolved my issue by switching the joblib backend to threading. Note, the call to register_parallel_backend lies outside the __init__ function.

from sklearn.utils import parallel_backend, register_parallel_backend

from joblib._parallel_backends import ThreadingBackend

class ModelTrainer(QRunnable):

register_parallel_backend('threading', ThreadingBackend, make_default=True)

def __init__(self, **kwargs):