重要概念
- 通信:sharing memory & message passing
- 架构:client-server & peer-to-peer
- Synchronization: bulk synchronous & asynchronous
- Parallelism: data parallelism ( more popular) VS model parallelism
并行编程模型
- MapReduce: message passing, client-server,and synchronous
- Parameter Server:message passing,client-server,and asynchronous
- Decentralized:message passing,peer-to-peer,synchronous or asynchronous
通信方式
- share memory
- Message passing
Synchronization
-
bulk synchronous
-
asynchronous
?? -
synchronous algorithm
-
asynchronous algorithm
编程模型
- MapReduce
- Parameter Server
- 去中心化网络
学习来源:https://www.bilibili.com/video/BV1zE411W7Wo?p=1