作者:rainoftime
链接:https://www.zhihu.com/question/30026369/answer/46528616
来源:知乎
著作权归作者所有。商业转载请联系作者获得授权,非商业转载请注明出处。
链接:https://www.zhihu.com/question/30026369/answer/46528616
来源:知乎
著作权归作者所有。商业转载请联系作者获得授权,非商业转载请注明出处。
1. 背景知识
体系结构
系统和网络
通信:RPC、RMI、MOM。。
进程和线程:
用户态、内核态;轻量级进程;协程;Actor。。
分布式相关问题
同步和互斥:保证相互冲突的并发进程可以共享资源
Double checked Locking、Immutable Value、Future 。。
事件分离和分发:Reactor、Proactor。。。
选举:从进程集中选出一个进程执行特别的任务
2. 分布式理论
数据结构
B树
log merge tree
merkle tree
一致性hash
DHT
vector clock
lock-free data structure
....
CAP、BASE
CAP: Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web
BASE an Acid Alternative
状态、时序
2PC、3PC 、Paxos ...
A brief history of Consensus- 2PC and Transaction Commit
Paxos Made Simple.
Paxos Made Practical
Paxos made live . An engineering perspective.
一致性、事务
Life beyond Distributed Transactions: an Apostate’s Opinion
Impossibility of distributed consensus with one faulty process.
Consensus on Transaction Commit.
Uniform consensus is harder than consensus
3. 分布式系统
分布式基础设施
消息队列
RabbitMQ 、ZeroMQ...
分布式锁服务、协调
The Chubby lock service for loosely-coupled distributed systems
Zookeeper
集群Monitoring
分布式存储系统
分布式文件系统
The Google file system.
Lustre
Cepth
Panasas
分布式块存储
Sheepdob
Parallax
Petal
分布式k-v存储系统
Dynamo: Amazon’s highly available key-value store
分布式表格系统
Amazon DynamoDB
Bigtable: A Distributed Storage System for Structured Data.
分布式数据库
Spanner: Google's Globally-Distributed Database
分布式计算
Map-Reduce
MapReduce: Simplified Data Processing on Large Clusters
内存计算
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing
流式计算
S4: Distributed Stream Computing Platform
Twitter Storm
图计算
GraphLab: A New Framework for Parallel Machine Learning
Pregel: a system for large-scale graph processing
4. 分布式应用
图片、视频等
Finding a Needle in Haystack: Facebook's Photo Storage
搜索
Web search for a planet: The Google cluster architecture
IM
体系结构
系统和网络
通信:RPC、RMI、MOM。。
进程和线程:
用户态、内核态;轻量级进程;协程;Actor。。
分布式相关问题
同步和互斥:保证相互冲突的并发进程可以共享资源
Double checked Locking、Immutable Value、Future 。。
事件分离和分发:Reactor、Proactor。。。
选举:从进程集中选出一个进程执行特别的任务
2. 分布式理论
数据结构
B树
log merge tree
merkle tree
一致性hash
DHT
vector clock
lock-free data structure
....
CAP、BASE
CAP: Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web
BASE an Acid Alternative
状态、时序
Time Clocks and the Ordering of Events in a Distributed System
Virtual Time and Global States of Distributed Systems
Distributed Snapshots: Determining Global States of a Distributed System
2PC、3PC 、Paxos ...
A brief history of Consensus- 2PC and Transaction Commit
Paxos Made Simple.
Paxos Made Practical
Paxos made live . An engineering perspective.
一致性、事务
Life beyond Distributed Transactions: an Apostate’s Opinion
Impossibility of distributed consensus with one faulty process.
Consensus on Transaction Commit.
Uniform consensus is harder than consensus
3. 分布式系统
分布式基础设施
消息队列
RabbitMQ 、ZeroMQ...
分布式锁服务、协调
The Chubby lock service for loosely-coupled distributed systems
Zookeeper
集群Monitoring
The ganglia distributed monitoring system:design, implementation, and
experience
Chukwa: A large-scale monitoring system
分布式存储系统
分布式文件系统
The Google file system.
Lustre
Cepth
Panasas
分布式块存储
Sheepdob
Parallax
Petal
分布式k-v存储系统
Dynamo: Amazon’s highly available key-value store
分布式表格系统
Amazon DynamoDB
Bigtable: A Distributed Storage System for Structured Data.
分布式数据库
Spanner: Google's Globally-Distributed Database
分布式计算
Map-Reduce
MapReduce: Simplified Data Processing on Large Clusters
内存计算
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing
流式计算
S4: Distributed Stream Computing Platform
Twitter Storm
图计算
GraphLab: A New Framework for Parallel Machine Learning
Pregel: a system for large-scale graph processing
4. 分布式应用
图片、视频等
Finding a Needle in Haystack: Facebook's Photo Storage
搜索
Web search for a planet: The Google cluster architecture
IM