Burrow, Validator
Validators are needed by tendermint core.
Tendermint是一个易于理解的BFT共识协议。协议遵循一个简单的状态机,如下:
协议中有两个角色:
(1)验证人:协议中的角色或者节点,不同的验证者在投票过程中具备不同的权力(vote power)。
(2)提议人:由验证人轮流产生。
验证人轮流对交易的区块提议并对提议的区块投票。区块被提交到链上,且每个区块就是一个区块高度。但区块也有可能提交失败,这种情况下协议将选择下一个验证人在相同高度上提议一个新块,重新开始投票。
Burrowed from: https://www.jianshu.com/p/68fb29cd00de
从图中可以看到,成功提交一个区块,必须经过两阶段的投票,称为pre-vote和pre-commit。当超过 2/3 的验证人在同一轮提议中对同一个块进行了pre-commit投票,那么这个区块才会被提交。
由于离线或者网络延迟等原因,可能造成提议人提议区块失败。这种情况在Tendermint中也是允许的,因为验证人会在进入下一轮提议之前等待一定时间,用于接收提议人提议的区块。
假设少于三分之一的验证人是拜占庭节点,Tendermint能够保证验证人永远不会在同一高度重复提交区块而造成冲突。为了做到这一点,Tendermint 引入了锁定机制,一旦验证人预投票了一个区块,那么该验证人就会被锁定在这个区块。然后:
(1)该验证人必须在预提交的区块进行预投票。
(2)当前一轮预提议和预投票没成功提交区块时,该验证人就会被解锁,然后进行对新块的下一轮预提交。
Genesis
Validators can be specified in genesisy doc:
[[GenesisDoc.Validators]]
Address = "728BA13EF4E5A15C147A0B891FFA36E96BD040BB"
PublicKey = "{\"CurveType\":\"ed25519\",\"PublicKey\":\"B0BAB4A9B89B78AC316E5348D8A48BAF87426415FCE25C6D2DDB4DDAB041CB25\"}"
Amount = 9999999998
Name = "Validator_0"
[[GenesisDoc.Validators.UnbondTo]]
Address = "728BA13EF4E5A15C147A0B891FFA36E96BD040BB"
PublicKey = "{\"CurveType\":\"ed25519\",\"PublicKey\":\"B0BAB4A9B89B78AC316E5348D8A48BAF87426415FCE25C6D2DDB4DDAB041CB25\"}"
Amount = 9999999998
Acctually, in order to starta node, a validator has to be specified, f.g.,
burrow start -v0 // to start burrow with validator set to the first one in genesis.
Loading Validator
On startup, burrow will create a in-memory validator from the address speficed in cmd line or config file, as shown below:
privVal, err := kern.PrivValidator(*conf.ValidatorAddress)
if err != nil {
return nil, fmt.Errorf("could not form PrivValidator from Address: %v", err)
}
// Generates an in-memory Tendermint PrivValidator (suitable for passing to LoadTendermintFromConfig)
func (kern *Kernel) PrivValidator(validator crypto.Address) (tmTypes.PrivValidator, error) {
val, err := keys.AddressableSigner(kern.keyClient, validator)
if err != nil {
return nil, fmt.Errorf("could not get validator addressable from keys client: %v", err)
}
signer, err := keys.AddressableSigner(kern.keyClient, val.GetAddress())
if err != nil {
return nil, err
}
return tendermint.NewPrivValidatorMemory(val, signer), nil
}
// PrivValidator defines the functionality of a local Tendermint validator
// that signs votes and proposals, and never double signs.
type PrivValidator interface {
// TODO: Extend the interface to return errors too. Issue: https://github.com/tendermint/tendermint/issues/3602
GetPubKey() crypto.PubKey
SignVote(chainID string, vote *Vote) error
SignProposal(chainID string, proposal *Proposal) error
}
PrivValidatorMemory implements the PrivValidator interface. This validator will be passed to tendermint later.
In Tendermint
This PrivValidator will be passed to ConsensusState in the end:
// Make ConsensusReactor
consensusReactor, consensusState := createConsensusReactor(
config, state, blockExec, blockStore, mempool, evidencePool,
privValidator, csMetrics, fastSync, eventBus, consensusLogger,
)
github.com/tendermint/tendermint/consensus.(*State).SetPrivValidator at state.go:258
github.com/tendermint/tendermint/node.createConsensusReactor at node.go:399
github.com/tendermint/tendermint/node.NewNode at node.go:656
github.com/hyperledger/burrow/consensus/tendermint.NewNode at tendermint.go:61
github.com/hyperledger/burrow/core.(*Kernel).LoadTendermintFromConfig at config.go:91
github.com/hyperledger/burrow/core.LoadKernelFromConfig at config.go:130
github.com/hyperledger/burrow/cmd/burrow/commands.Start.func1.1 at start.go:26
github.com/jawher/mow.cli/internal/flow.(*Step).callDo at flow.go:55
github.com/jawher/mow.cli/internal/flow.(*Step).Run at flow.go:25
github.com/jawher/mow.cli/internal/flow.(*Step).Run at flow.go:29
github.com/jawher/mow.cli/internal/flow.(*Step).Run at flow.go:29
github.com/jawher/mow.cli/internal/flow.(*Step).Run at flow.go:29
github.com/jawher/mow%2ecli.(*Cmd).parse at commands.go:681
github.com/jawher/mow%2ecli.(*Cmd).parse at commands.go:695
github.com/jawher/mow%2ecli.(*Cli).parse at cli.go:76
github.com/jawher/mow%2ecli.(*Cli).Run at cli.go:105
main.main at main.go:15
runtime.main at proc.go:203
runtime.goexit at asm_amd64.s:1357
- Async stack trace
runtime.rt0_go at asm_amd64.s:220
Validator Ring
Take a look at the call stack:
github.com/hyperledger/burrow/execution/state.LoadValidatorRing at validators.go:27
github.com/hyperledger/burrow/execution/state.LoadState at state.go:214
github.com/hyperledger/burrow/core.(*Kernel).LoadState at kernel.go:118
github.com/hyperledger/burrow/core.LoadKernelFromConfig at config.go:116
github.com/hyperledger/burrow/cmd/burrow/commands.Start.func1.1 at start.go:26
github.com/jawher/mow.cli/internal/flow.(*Step).callDo at flow.go:55
github.com/jawher/mow.cli/internal/flow.(*Step).Run at flow.go:25
github.com/jawher/mow.cli/internal/flow.(*Step).Run at flow.go:29
github.com/jawher/mow.cli/internal/flow.(*Step).Run at flow.go:29
github.com/jawher/mow.cli/internal/flow.(*Step).Run at flow.go:29
github.com/jawher/mow%2ecli.(*Cmd).parse at commands.go:681
github.com/jawher/mow%2ecli.(*Cmd).parse at commands.go:695
github.com/jawher/mow%2ecli.(*Cli).parse at cli.go:76
github.com/jawher/mow%2ecli.(*Cli).Run at cli.go:105
main.main at main.go:15
runtime.main at proc.go:203
runtime.goexit at asm_amd64.s:1357
- Async stack trace
runtime.rt0_go at asm_amd64.s:220
type State struct {
sync.Mutex
db dbm.DB
ReadState
writeState writeState
logger *logging.Logger
}
type ReadState struct {
Forest storage.ForestReader
Plain *storage.PrefixDB
validator.History
}
type writeState struct {
forest *storage.MutableForest
plain *storage.PrefixDB
ring *validator.Ring
accountStats acmstate.AccountStats
nodeStats registry.NodeStats
}
When loading state from DB, it will try to LoadValidatorRing…
// Tries to load the execution state from DB, returns nil with no error if no state found
func LoadState(db dbm.DB, version int64) (*State, error) {
s := NewState(db)
err := s.writeState.forest.Load(version)
// Populate stats. If this starts taking too long, store the value rather than the full scan at startup
err = s.loadAccountStats()
err = s.loadNodeStats()
// load the validator ring
ring, err := LoadValidatorRing(version, DefaultValidatorsWindowSize, s.writeState.forest.GetImmutable)
s.writeState.ring = ring
s.ReadState.History = ring
return s, nil
}
// Initialises the validator Ring from the validator storage in forest
func LoadValidatorRing(version int64, ringSize int,
getImmutable func(version int64) (*storage.ImmutableForest, error)) (*validator.Ring, error) {
// In this method we have to page through previous version of the tree in order to reconstruct the in-memory
// ring structure. The corner cases are a little subtle but printing the buckets helps
// The basic idea is to load each version of the tree ringSize back, work out the difference that must have occurred
// between each bucket in the ring, and apply each diff to the ring. Once the ring is full it is symmetrical (up to
// a reindexing). If we are loading a chain whose height is less than the ring size we need to get the initial state
// correct
startVersion := version - int64(ringSize)
if startVersion < 1 {
// The ring will not be fully populated
startVersion = 1
}
var err error
// Read state to pull immutable forests from
rs := &ReadState{}
// Start with an empty ring - we want the initial bucket to have no cumulative power
ring := validator.NewRing(nil, ringSize)
// Load the IAVL state
rs.Forest, err = getImmutable(startVersion)
if err != nil {
return nil, err
}
// Write the validator state at startVersion from IAVL tree into the ring's current bucket delta
err = validator.Write(ring, rs)
if err != nil {
return nil, err
}
// Rotate, now we have [ {bucket 0: cum: {}, delta: {start version changes} }, {bucket 1: cum: {start version changes}, delta {}, ... ]
// which is what we need (in particular we need this initial state if we are loading into a incompletely populated ring
_, _, err = ring.Rotate()
if err != nil {
return nil, err
}
// Rebuild validator Ring
for v := startVersion + 1; v <= version; v++ {
// Update IAVL read state to version of interest
rs.Forest, err = getImmutable(v)
if err != nil {
return nil, err
}
// Calculate the difference between the rings current cum and what is in state at this version
diff, err := validator.Diff(ring.CurrentSet(), rs)
if err != nil {
return nil, err
}
// Write that diff into the ring (just like it was when it was originally written to setPower)
err = validator.Write(ring, diff)
if err != nil {
return nil, err
}
// Rotate just like it was on the original commit
_, _, err = ring.Rotate()
if err != nil {
return nil, err
}
}
// Our ring should be the same up to symmetry in its index so we reindex to regain equality with the version we are loading
// This is the head index we would have had if we had started from version 1 like the chain did
ring.ReIndex(int(version % int64(ringSize)))
return ring, err
}
It will initialize the validator Ring from the validator storage in forest.
// The basic idea is to load each version of the tree ringSize back, work out the difference that must have occurred
// between each bucket in the ring, and apply each diff to the ring. Once the ring is full it is symmetrical (up to
// a reindexing). If we are loading a chain whose height is less than the ring size we need to get the initial state
// correct
Ring rotation
Validator ring will rotate to advance when tendermint invokes ACBI.CommitSync, which will ask burrow State ro commit the changes:
github.com/hyperledger/burrow/acm/validator.(*Ring).Rotate at ring.go:92
github.com/hyperledger/burrow/execution/state.(*State).commit at state.go:286
github.com/hyperledger/burrow/execution/state.(*State).Update at state.go:277
github.com/hyperledger/burrow/execution.(*executor).Commit at execution.go:359
github.com/hyperledger/burrow/consensus/abci.(*App).Commit at app.go:295
github.com/tendermint/tendermint/abci/client.(*localClient).CommitSync at local_client.go:215
github.com/tendermint/tendermint/proxy.(*appConnConsensus).CommitSync at app_conn.go:81
github.com/tendermint/tendermint/state.(*BlockExecutor).Commit at execution.go:212
github.com/tendermint/tendermint/state.(*BlockExecutor).ApplyBlock at execution.go:166
github.com/tendermint/tendermint/consensus.(*State).finalizeCommit at state.go:1431
github.com/tendermint/tendermint/consensus.(*State).tryFinalizeCommit at state.go:1350
github.com/tendermint/tendermint/consensus.(*State).enterCommit.func1 at state.go:1285
github.com/tendermint/tendermint/consensus.(*State).enterCommit at state.go:1322
github.com/tendermint/tendermint/consensus.(*State).addVote at state.go:1819
github.com/tendermint/tendermint/consensus.(*State).tryAddVote at state.go:1642
github.com/tendermint/tendermint/consensus.(*State).handleMsg at state.go:709
github.com/tendermint/tendermint/consensus.(*State).receiveRoutine at state.go:660
runtime.goexit at asm_amd64.s:1357
- Async stack trace
github.com/tendermint/tendermint/consensus.(*State).OnStart at state.go:335
func (s *State) commit() ([]byte, int64, error) {
// save state at a new version may still be orphaned before we save the version against the hash
hash, version, err := s.writeState.forest.Save()
if err != nil {
return nil, 0, err
}
totalPowerChange, totalFlow, err := s.writeState.ring.Rotate()
if err != nil {
return nil, 0, err
}
if totalFlow.Sign() != 0 {
//noinspection ALL
s.logger.InfoMsg("validator set changes", "total_power_change", totalPowerChange, "total_flow", totalFlow)
}
return hash, version, err
}
// Rotate the current head bucket to the next bucket and returns the change in total power between the previous bucket
// and the current head, and the total flow which is the sum of absolute values of all changes each validator's power
// after rotation the next head is a copy of the current head
func (vc *Ring) Rotate() (totalPowerChange *big.Int, totalFlow *big.Int, err error) {
// Subtract the tail bucket (if any) from the total
err = Subtract(vc.power, vc.Next().Delta)
if err != nil {
return
}
// Capture current head as previous before advancing buffer
prevHead := vc.Head()
// Add head delta to total power
err = Add(vc.power, prevHead.Delta)
if err != nil {
return
}
// Advance the ring buffer
vc.head = vc.index(1)
// Overwrite new head bucket (previous tail) with a fresh bucket with Previous_i+1 = Next_i = Previous_i + Delta_i
vc.buckets[vc.head] = NewBucket(prevHead.Next)
// Capture flow before we wipe it
totalFlow = prevHead.Flow.totalPower
// Subtract the previous bucket total power so we can add on the current buckets power after this
totalPowerChange = new(big.Int).Sub(vc.Head().Previous.TotalPower(), prevHead.Previous.TotalPower())
// Record how many of our buckets we have cycled over
if vc.populated < vc.size {
vc.populated++
}
return
}
BeginBlock
It will do some checks on the validators. Note the validatorSet is a bit difference between burrow and tendermint. Tendermint is 2 blocks behind burrow.
func (app *App) BeginBlock(block types.RequestBeginBlock) (respBeginBlock types.ResponseBeginBlock) {
app.block = &block
defer func() {
if r := recover(); r != nil {
app.panicFunc(fmt.Errorf("panic occurred in abci.App/BeginBlock: %v\n%s", r, debug.Stack()))
}
}()
if block.Header.Height > 1 {
var err error
previousValidators := validator.NewTrimSet()
// Tendermint runs two blocks behind plus we are updating in end block validators updated last round
err = validator.Write(previousValidators,
app.validators.Validators(BurrowValidatorDelayInBlocks+TendermintValidatorDelayInBlocks))
if err != nil {
panic(fmt.Errorf("could not build current validator set: %v", err))
}
if len(block.LastCommitInfo.Votes) != previousValidators.Size() {
err = fmt.Errorf("Tendermint passes %d validators to BeginBlock but Burrow's has %d:\n %v",
len(block.LastCommitInfo.Votes), previousValidators.Size(), previousValidators.String())
panic(err)
}
for _, v := range block.LastCommitInfo.Votes {
err = app.checkValidatorMatches(previousValidators, v.Validator)
if err != nil {
panic(err)
}
}
}
return
}
EndBlock
It offers a chance for Burrow/APP to update the validator’s information. ValidatorUpdates as the returned value will be handled by tendermint.
func (app *App) EndBlock(reqEndBlock types.RequestEndBlock) types.ResponseEndBlock {
var validatorUpdates []types.ValidatorUpdate
defer func() {
if r := recover(); r != nil {
app.panicFunc(fmt.Errorf("panic occurred in abci.App/EndBlock: %v\n%s", r, debug.Stack()))
}
}()
err := app.validators.ValidatorChanges(BurrowValidatorDelayInBlocks).IterateValidators(func(id crypto.Addressable, power *big.Int) error {
app.logger.InfoMsg("Updating validator power", "validator_address", id.GetAddress(),
"new_power", power)
validatorUpdates = append(validatorUpdates, types.ValidatorUpdate{
PubKey: id.GetPublicKey().ABCIPubKey(),
// Must ensure power fits in an int64 during execution
Power: power.Int64(),
})
return nil
})
if err != nil {
panic(err)
}
return types.ResponseEndBlock{
ValidatorUpdates: validatorUpdates,
}
}
Tendermint ValidatorSet
ValidatorSet is maintaied by Consensus.RoundState:
// RoundState defines the internal consensus state.
// NOTE: Not thread safe. Should only be manipulated by functions downstream
// of the cs.receiveRoutine
type RoundState struct {
Height int64 `json:"height"` // Height we are working on
Round int `json:"round"`
Step RoundStepType `json:"step"`
StartTime time.Time `json:"start_time"`
// Subjective time when +2/3 precommits for Block at Round were found
CommitTime time.Time `json:"commit_time"`
Validators *types.ValidatorSet `json:"validators"`
Proposal *types.Proposal `json:"proposal"`
ProposalBlock *types.Block `json:"proposal_block"`
ProposalBlockParts *types.PartSet `json:"proposal_block_parts"`
LockedRound int `json:"locked_round"`
LockedBlock *types.Block `json:"locked_block"`
LockedBlockParts *types.PartSet `json:"locked_block_parts"`
// Last known round with POL for non-nil valid block.
ValidRound int `json:"valid_round"`
ValidBlock *types.Block `json:"valid_block"` // Last known block of POL mentioned above.
// Last known block parts of POL metnioned above.
ValidBlockParts *types.PartSet `json:"valid_block_parts"`
Votes *HeightVoteSet `json:"votes"`
CommitRound int `json:"commit_round"` //
LastCommit *types.VoteSet `json:"last_commit"` // Last precommits at Height-1
LastValidators *types.ValidatorSet `json:"last_validators"`
TriggeredTimeoutPrecommit bool `json:"triggered_timeout_precommit"`
}
// ValidatorSet represent a set of *Validator at a given height.
// The validators can be fetched by address or index.
// The index is in order of .Address, so the indices are fixed
// for all rounds of a given blockchain height - ie. the validators
// are sorted by their address.
// On the other hand, the .ProposerPriority of each validator and
// the designated .GetProposer() of a set changes every round,
// upon calling .IncrementProposerPriority().
// NOTE: Not goroutine-safe.
// NOTE: All get/set to validators should copy the value for safety.
type ValidatorSet struct {
// NOTE: persisted via reflect, must be exported.
Validators []*Validator `json:"validators"`
Proposer *Validator `json:"proposer"`
// cached (unexported)
totalVotingPower int64
}
// Volatile state for each Validator
// NOTE: The ProposerPriority is not included in Validator.Hash();
// make sure to update that method if changes are made here
type Validator struct {
Address Address `json:"address"`
PubKey crypto.PubKey `json:"pub_key"`
VotingPower int64 `json:"voting_power"`
ProposerPriority int64 `json:"proposer_priority"`
}
Example
Validators: ValidatorSet{
Proposer: Validator{D37EF35CD7322E109F3C3B940209E81BF97E76EA PubKeyEd25519{6921E42DAA94B59EFACBD7B1BDFC055BADC0C6BAA8A13CB38A4D922585118B5F} VP:9999999999 A:5624965624}
Validators:
Validator{8C3790592D35D4B865893D5FAE9EED594E1A7902 PubKeyEd25519{18622C47285F3CC85C909C042E50EF9C9C757A7EF2ECB2F18FE6CE886741AF17} VP:10000 A:-5624965624}
Validator{D37EF35CD7322E109F3C3B940209E81BF97E76EA PubKeyEd25519{6921E42DAA94B59EFACBD7B1BDFC055BADC0C6BAA8A13CB38A4D922585118B5F} VP:9999999999 A:5624965624}
}
Burrow Update Validator
Tendermint executes block txs by running ApplyBlock(). It will invoke Burrow’s callbacks:
abciResponses, err := execBlockOnProxyApp(blockExec.logger, blockExec.proxyApp, block, blockExec.db)
Therefore, ACBI BeginBlock and EndBlock will be invoked in sequence. In EndBlock, anychanges will be returned in:
types.ResponseEndBlock{
ValidatorUpdates: validatorUpdates,
}
which will be further processed by tendermint’s State.
// ApplyBlock validates the block against the state, executes it against the app,
// fires the relevant events, commits the app, and saves the new state and responses.
// It's the only function that needs to be called
// from outside this package to process and commit an entire block.
// It takes a blockID to avoid recomputing the parts hash.
func (blockExec *BlockExecutor) ApplyBlock(state State, blockID types.BlockID, block *types.Block) (State, error) {
if err := blockExec.ValidateBlock(state, block); err != nil {
return state, ErrInvalidBlock(err)
}
startTime := time.Now().UnixNano()
// Callback to burrow
abciResponses, err := execBlockOnProxyApp(blockExec.logger, blockExec.proxyApp, block, blockExec.db)
endTime := time.Now().UnixNano()
blockExec.metrics.BlockProcessingTime.Observe(float64(endTime-startTime) / 1000000)
if err != nil {
return state, ErrProxyAppConn(err)
}
fail.Fail() // XXX
// Save the results before we commit.
SaveABCIResponses(blockExec.db, block.Height, abciResponses)
fail.Fail() // XXX
// validate the validator updates and convert to tendermint types
abciValUpdates := abciResponses.EndBlock.ValidatorUpdates
err = validateValidatorUpdates(abciValUpdates, state.ConsensusParams.Validator)
if err != nil {
return state, fmt.Errorf("error in validator updates: %v", err)
}
validatorUpdates, err := types.PB2TM.ValidatorUpdates(abciValUpdates)
if err != nil {
return state, err
}
if len(validatorUpdates) > 0 {
blockExec.logger.Info("Updates to validators", "updates", types.ValidatorListString(validatorUpdates))
}
// Update the state with the block and responses.
state, err = updateState(state, blockID, &block.Header, abciResponses, validatorUpdates)
if err != nil {
return state, fmt.Errorf("commit failed for application: %v", err)
}
// Lock mempool, commit app state, update mempoool.
appHash, err := blockExec.Commit(state, block, abciResponses.DeliverTxs)
if err != nil {
return state, fmt.Errorf("commit failed for application: %v", err)
}
// Update evpool with the block and state.
blockExec.evpool.Update(block, state)
fail.Fail() // XXX
// Update the app hash and save the state.
state.AppHash = appHash
SaveState(blockExec.db, state)
fail.Fail() // XXX
// Events are fired after everything else.
// NOTE: if we crash between Commit and Save, events wont be fired during replay
fireEvents(blockExec.logger, blockExec.eventBus, block, abciResponses, validatorUpdates)
return state, nil
}
// Executes block's transactions on proxyAppConn.
// Returns a list of transaction results and updates to the validator set
func execBlockOnProxyApp(
logger log.Logger,
proxyAppConn proxy.AppConnConsensus,
block *types.Block,
stateDB dbm.DB,
) (*ABCIResponses, error) {
var validTxs, invalidTxs = 0, 0
txIndex := 0
abciResponses := NewABCIResponses(block)
// Execute transactions and get hash.
proxyCb := func(req *abci.Request, res *abci.Response) {
if r, ok := res.Value.(*abci.Response_DeliverTx); ok {
// TODO: make use of res.Log
// TODO: make use of this info
// Blocks may include invalid txs.
txRes := r.DeliverTx
if txRes.Code == abci.CodeTypeOK {
validTxs++
} else {
logger.Debug("Invalid tx", "code", txRes.Code, "log", txRes.Log)
invalidTxs++
}
abciResponses.DeliverTxs[txIndex] = txRes
txIndex++
}
}
proxyAppConn.SetResponseCallback(proxyCb)
commitInfo, byzVals := getBeginBlockValidatorInfo(block, stateDB)
// Begin block
var err error
abciResponses.BeginBlock, err = proxyAppConn.BeginBlockSync(abci.RequestBeginBlock{
Hash: block.Hash(),
Header: types.TM2PB.Header(&block.Header),
LastCommitInfo: commitInfo,
ByzantineValidators: byzVals,
})
if err != nil {
logger.Error("Error in proxyAppConn.BeginBlock", "err", err)
return nil, err
}
// Run txs of block.
for _, tx := range block.Txs {
proxyAppConn.DeliverTxAsync(abci.RequestDeliverTx{Tx: tx})
if err := proxyAppConn.Error(); err != nil {
return nil, err
}
}
// End block.
abciResponses.EndBlock, err = proxyAppConn.EndBlockSync(abci.RequestEndBlock{Height: block.Height})
if err != nil {
logger.Error("Error in proxyAppConn.EndBlock", "err", err)
return nil, err
}
logger.Info("Executed block", "height", block.Height, "validTxs", validTxs, "invalidTxs", invalidTxs)
return abciResponses, nil
}
github.com/tendermint/tendermint/types.(*ValidatorSet).applyUpdates at validator_set.go:474
github.com/tendermint/tendermint/types.(*ValidatorSet).updateWithChangeSet at validator_set.go:599
github.com/tendermint/tendermint/types.(*ValidatorSet).UpdateWithChangeSet at validator_set.go:624
github.com/tendermint/tendermint/state.updateState at execution.go:397
github.com/tendermint/tendermint/state.(*BlockExecutor).ApplyBlock at execution.go:160
github.com/tendermint/tendermint/consensus.(*State).finalizeCommit at state.go:1431
github.com/tendermint/tendermint/consensus.(*State).tryFinalizeCommit at state.go:1350
github.com/tendermint/tendermint/consensus.(*State).addProposalBlockPart at state.go:1633
github.com/tendermint/tendermint/consensus.(*State).handleMsg at state.go:690
github.com/tendermint/tendermint/consensus.(*State).receiveRoutine at state.go:644
runtime.goexit at asm_amd64.s:1357
- Async stack trace
github.com/tendermint/tendermint/consensus.(*State).OnStart at state.go:335