python分布式计算dispy简单使用_distpy 分布式-CSDN博客

本文链接：https://blog.csdn.net/suzyu12345/article/details/53909688

dispy是一个用于分布式并行计算的Python框架，由JobCluster和SharedJobCluster组成。通常，仅需dispy和dispynode即可满足需求。在服务器端启动dispynode以接收和处理计算任务，客户端则需注意函数导入方式。计算节点的模块导入应在函数内部，自定义模块需额外处理。更多信息参考官方文档和相关博客。

摘要由CSDN通过智能技术生成

dispy，是用asyncoro实现的分布式并行计算框架。

框架也是非常精简，只有4个组件，在其源码文件夹下可以找到：

dispy.py (client) provides two ways of creating “clusters”: JobCluster when only one instance of dispy may run and SharedJobCluster when multiple instances may run (in separate processes). If JobCluster is used, the scheduler contained within dispy.py will distribute jobs on the server nodes; if SharedJobCluster is used, a separate scheduler (dispyscheduler) must be running.

dispynode.py executes jobs on behalf of dispy. dispynode must be running on each of the (server) nodes that form the cluster.

dispyscheduler.py is needed only when SharedJobCluster is used; this provides a scheduler that can be shared by multiple dispy users.

dispynetrelay.py is needed when nodes are located across different networks; this relays information about nodes on a network to the scheduler. If all the nodes are on same network, there is no need for dispynetrelay - the scheduler and nodes automatically discover each other.

一般情况下，使用dispy和dispynode就已经足够解决问题了。

简单使用：

服务器端：
在服务器端启动dispy，监听并接收所有发来的计算任务，完成计算后将结果返回给客户端。
打开python_home/Scripts文件夹，在安装dispy后会有上面说到的4个dispy组件，以py文件形式存在。当然你也可以在dispy的源码文件夹