分布式一致性协议之Raft的实现详解

到目前为止，不管是哪门语言，应该都已经有一些raft协议的实现了。但是大家的实现也都是根据raft协议论文来的，根据自己的服务形态在细节上有一些差异而已，大体上是一样的。

因此今天这里就以golang语言中的raft实现库为例，进行剖析，让大家实际感受下raft的流程。

hashicorp/raft 是常用的 Golang 版 Raft 算法的实现，被众多流行软件使用，如 Consul、InfluxDB、IPFS 等 

raft.go 是 Hashicorp Raft 的核心代码文件，大部分的核心功能都是在这个文件中实现。 

在 Hashicorp Raft 中，支持两种节点间通讯机制，内存型和 TCP 协议型，其中，内存型通讯机制，主要用于测试，2 种通讯机制的代码实现，分别在文件 inmem_transport.go 和 tcp_transport.go 中。

概念与术语

• leader：领导者，提供服务(生成写日志)的节点，任何时候raft系统中只能有一个leader。

• follower：跟随者，被动接受请求的节点，不会发送任何请求，只会响应来自leader或者candidate的请求。如果接受到客户请求，会转发给leader。

• candidate：候选人，选举过程中产生，follower在超时时间内没有收到leader的心跳或者日志，则切换到candidate状态，进入选举流程。

• termId：任期号，时间被划分成一个个任期，每次选举后都会产生一个新的termId，一个任期内只有一个leader。

• RequestVote：请求投票，candidate在选举过程中发起，收到quorum(多数派）响应后，成为leader。

• AppendEntries：附加日志，leader发送日志和心跳的机制

多数派计算:

func (r *Raft) quorumSize() int {
  voters := 0
  for _, server := range r.configurations.latest.Servers {
    if server.Suffrage == Voter {
      voters++
    }
  }
  return voters/2 + 1
}

Raft节点运行

func (r *Raft) run() {
  for {
    // Check if we are doing a shutdown
    select {
    case <-r.shutdownCh:
      // Clear the leader to prevent forwarding
      r.setLeader("")
      return
    default:
    }


    // Enter into a sub-FSM
    switch r.getState() {
    case Follower:
      r.runFollower()
    case Candidate:
      r.runCandidate()
    case Leader:
      r.runLeader()
    }
  }
}

跟随者、候选人、领导者 3 种节点状态都有分别对应的功能函数

创建Raft节点

每次服务起来的时候，都是创建一个raft节点：

func NewRaft(conf *Config, fsm FSM, logs LogStore, stable StableStore, snaps SnapshotStore, trans Transport) (*Raft, error) {
  // Validate the configuration.
  if err := ValidateConfig(conf); err != nil {
    return nil, err
  }


  // Ensure we have a LogOutput.
  var logger hclog.Logger
  if conf.Logger != nil {
    logger = conf.Logger
  } else {
    if conf.LogOutput == nil {
      conf.LogOutput = os.Stderr
    }


    logger = hclog.New(&hclog.LoggerOptions{
      Name:   "raft",
      Level:  hclog.LevelFromString(conf.LogLevel),
      Output: conf.LogOutput,
    })
  }


  // Try to restore the current term.
  currentTerm, err := stable.GetUint64(keyCurrentTerm)
  if err != nil && err.Error() != "not found" {
    return nil, fmt.Errorf("failed to load current term: %v", err)
  }


  // Read the index of the last log entry.
  lastIndex, err := logs.LastIndex()
  if err != nil {
    return nil, fmt.Errorf("failed to find last log: %v", err)
  }


  // Get the last log entry.
  var lastLog Log
  if lastIndex > 0 {
    if err = logs.GetLog(lastIndex, &lastLog); err != nil {
      return nil, fmt.Errorf("failed to get last log at index %d: %v", lastIndex, err)
    }
  }


  // Make sure we have a valid server address and ID.
  protocolVersion := conf.ProtocolVersion
  localAddr := ServerAddress(trans.LocalAddr())
  localID := conf.LocalID


  // TODO (slackpad) - When we deprecate protocol version 2, remove this
  // along with the AddPeer() and RemovePeer() APIs.
  if protocolVersion < 3 && string(localID) != string(localAddr) {
    return nil, fmt.Errorf("when running with ProtocolVersion < 3, LocalID must be set to the network address")
  }


  // Create Raft struct.
  r := &Raft{
    protocolVersion:       protocolVersion,
    applyCh:               make(chan *logFuture),
    conf:                  *conf,
    fsm:                   fsm,
    fsmMutateCh:           make(chan interface{}, 128),
    fsmSnapshotCh:         make(chan *reqSnapshotFuture),
    leaderCh:              make(chan bool),
    localID:               localID,
    localAddr:             localAddr,
    logger:                logger,
    logs:                  logs,
    configurationChangeCh: make(chan *configurationChangeFuture),
    configurations:        configurations{},
    rpcCh:                 trans.Consumer(),
    snapshots:             snaps,
    userSnapshotCh:        make(chan *userSnapshotFuture),
    userRestoreCh:         make(chan *userRestoreFuture),
    shutdownCh:            make(chan struct{}),
    stable:                stable,
    trans:                 trans,
    verifyCh:              make(chan *verifyFuture, 64),
    configurationsCh:      make(chan *configurationsFuture, 8),
    bootstrapCh:           make(chan *bootstrapFuture),
    observers:             make(map[uint64]*Observer),
    leadershipTransferCh:  make(chan *leadershipTransferFuture, 1),
  }


  // Initialize as a follower.
  r.setState(Follower)


  // Start as leader if specified. This should only be used
  // for testing purposes.
  if conf.StartAsLeader {
    r.setState(Leader)
    r.setLeader(r.localAddr)
  }


  // Restore the current term and the last log.
  r.setCurrentTerm(currentTerm)
  r.setLastLog(lastLog.Index, lastLog.Term)


  // Attempt to restore a snapshot if there are any.
  if err := r.restoreSnapshot(); err != nil {
    return nil, err
  }


  // Scan through the log for any configuration change entries.
  snapshotIndex, _ := r.getLastSnapshot()
  for index := snapshotIndex + 1; index <= lastLog.Index; index++ {
    var entry Log
    if err := r.logs.GetLog(index, &entry); err != nil {
      r.logger.Error("failed to get log", "index", index, "error", err)
      panic(err)
    }
    r.processConfigurationLogEntry(&entry)
  }
  r.logger.Info("initial configuration",
    "index", r.configurations.latestIndex,
    "servers", hclog.Fmt("%+v", r.configurations.latest.Servers))


  // Setup a heartbeat fast-path to avoid head-of-line
  // blocking where possible. It MUST be safe for this
  // to be called concurrently with a blocking RPC.
  trans.SetHeartbeatHandler(r.processHeartbeat)


  if conf.skipStartup {
    return r, nil
  }
  // Start the background work.
  r.goFunc(r.run)
  r.goFu