Random Beacon Chain

Random Beacon Chain

Status: retired

Authors: Raul Jordan, Terence Tsao, Yutaro Mori, Preston Van Loon, Nishant Das

Prysmatic Labs

Last Updated: July 10, 2018 OUTDATED: See v2.1 design

Background

This is a design doc for implementation of the most basic iteration of a random beacon chain with its relevant actors. This doc will also analyze options for developing this system as a fresh repository (no dependencies on geth) and the best way to do so with our current code.

Goals

The most important goal is to design and implement a minimal random beacon chain and merge aspects of our current work into a new, functioning system that we can use for demonstration purposes. The other goals of this design will be to:

● Figure out the responsibilities of actors in the Random Beacon Chain architecture

● Determine what basic components we need for an MVP and which ones we can push out until later

 

This write up documents the design of beacon chain API. We will place the API in beacon-chain repo to replace SMC in geth-sharding with no-ops corresponding to an interface for beacon chain. By following the latest beacon chain spec, the main differences are shuffling method, advancing block, transitioning state and configuration options will be different.

 

What is included in the Ruby Release

● Rewards and penalties

● Processing PoW chain transactions

● Dynasty transition

○ In the spec includes changing of the validator set

● Simple source of randomness in the Ruby release

 

What’s not included in Ruby release?

● Processing shard chain collations

○ Simply attesters, proposers, but no notaries signing off on cross-links

● Check signature with BLS

Design

Shuffling Validators (Note: Changed by Latest Research)

The following algorithm shuffles the active validator set. The proposer and attesters for each block are determined by shuffling the validator set with active_state.randao as the seed.

 

1. Loop through all the validator position

2. Get a random 32 bytes seed

3. For every 3 bytes in 32 bytes seed, convert 3 bytes to int

4. Replace the current validator index with the index using converted int

 

while not_all_validator_positions_shuffled: source = get_pseudorandom_32_bytes(source) for every_three_bytes in source: m = convert_three_bytes_to_int(current_three_bytes) remaining = the_number_of_validators_left_to_shuffle if none_remaining: break replacement_pos = get_replacement_position_for_i(m, i) swap(i, replacement_pos) go_to_next_position() # i += 1

 

For our first release, a simple source of randomness derived from blockhashes may be enough.

Shuffling Notaries

Note: May not be included in Ruby release, but included here for completeness

 

The shards to crosslink are determined by selecting the first N shards that have been crosslinked least recently, where N is based on how many validators are available.

 

def get_crosslink_shards(crystallized_state): start_from = crystallized_state.next_shard count = len(crystallized_state.active_validators) // NOTARIES_PER_CROSSLINK if start_from + count <= SHARD_COUNT: return list(range(start_from, start_from + count)) else: return list(range(start_from, SHARD_COUNT)) + list(range(start_from + count - SHARD_COUNT))

Notary assignment for each shard is shuffled every dynasty transition by resetting crystallized_state.crosslink_seed (see Advancing Beacon Chain Blocks).

 

def get_crosslink_notaries(crystallized_state, shard_id): crosslink_shards = get_crosslink_shards(crystallized_state) if shard_id not in crosslink_shards: return None all = len(crystallized_state.current_shuffling) start = all * crosslink_shards.index(shard_id) // len(crosslink_shards) end = all * (crosslink_shards.index(shard_id)+1) // len(crosslink_shards) return crystallized_state.current_shuffling[start: end]

Advancing Beacon Chain Blocks

State is split into active_state and crystallized_state. active_state transitions occur every block and records validator votes. crystallized_state transitions occur every 100 blocks and handles shard crosslinking, justified and finalized blocks (Casper FFG), and updates to the validator set. However, the validator set is updated only when all shards have been crosslinked (Dynasty Transition).

Per-block Transition

● Update active_state.randao with randao_reveal

● Update active_state.ffg_voter_bitfield with:

○ The proposer that signed the block

○ The attesters from attestation_bitfield

○ Notaries that voted in shard_aggregate_vote

● Update recent_attesters with the attesters that voted for this block ● Update active_state.partial_crosslinks (full spec)

● Update active_state.recent_proposers with the proposer’s randao_reveal

● Increment active_state.total_skip_count by skip_count

● Increment active_state.height by 1

Epoch Transition

Every 100 blocks do the following:

● If ⅔ of all validators voted, crystallized_state.justified_epoch = crystallized_state.current_epoch

● If two consecutive checkpoints are justified, crystallized_state.finalized_epoch = crystallized_state.current_epoch

● If any PartialCrosslinkRecord from active_state.partial_crosslinks gets ⅔ of votes, update crystallized_state.crosslink_records

● Increment crystallized_state.current_epoch by 1

 

Note: Deposit balance changes are out of scope because the rules are undefined and transactions aren’t processed in

 

Note: Crosslink seed-related calculations are out of scope because handling shards that are slow to crosslink is non-critical.

Dynasty Transition

Occurs during:

● An epoch transition (once every 100 blocks)

● When all shards have been crosslinked since the last dynasty transition

 

Then the following updates are made:

● crystallized_state.dynasty += 1

● crystallized_state.crosslink_seed = active_state.randao

● Update the active, queued, and exited validators. Updating exited validator’s balance is out of this scope.

Fork Choice Rule for Beacon Chain

In addition to the Casper FFG fork choice rule to follow the chain containing the justified checkpoint of the great height, because the PoW chain is a side chain, an additional requirement is that main_chain_ref is inside the highest scoring PoW chain and that it is a descendent of the parent block’s main_chain_ref.

 

The head can be updated with the following algorithm:

def update_head(chain, old_head, new_head): a, b = old_head, new_head while a.height > b.height: a = get_parent(a) while b.height > a.height: b = get_parent(b) while a != b: a, b = get_parent(a), get_parent(b) b = new_head new_chain = [] while b.height > a.height: new_chain = [b] + new_chain b = get_parent(b) chain = chain[:a.height + 1] + new_chain

In the event of a reorg in the PoW chain, the beacon chain must be reorged with the following:

def reorg_beacon_chain(main_chain, beacon_chain): old_head = beacon_chain[-1]

while beacon_chain[-1].main_chain_ref not in main_chain: beacon_chain.pop() queue = [beacon_chain[-1]] new_head = beacon_chain[-1] while len(queue) > 0: b = queue.pop(0) for c in get_children(b): if c.main_chain_ref in main_chain: if get_score(c) > get_score(new_head): new_head = c queue.append(c) update_head(beacon_chain, old_head, new_head)

Beacon Chain P2P

Shard p2p has to accommodate quick reshuffling across shards that live in different networks and has to incorporate different subprotocols for sharding. We can treat the beacon chain nodes as their own network (perhaps shard -1) and use the same floodsub mechanism.

 

The steps required are the same as before for shardp2p:

● Use gossipsub for rapidly switching across shard networks

● Define shard subprotocols ● Torus-shaped shard discovery protocol: see ETHResearch

Beacon Node Sync

There are two steps required for beacon node sync

1. Initial chain sync via p2p which will fetch the latest block number from the network and connect to other beacon node peers that will provide the history of the beacon chain

2. After head is reached, the sync to listen for new blocks being posted from the network begins

Actors in the Random Beacon Chain

Beacon Chain Node

● A beacon chain node is responsible for running the entire beacon chain system, listening for main chain block checkpoints via an RPC connection to a running Geth node, downloading the beacon chain history, and advancing the canonical chain via Casper PoS.

● Key Functions

○ Initializing a node with service lifecycle management

○ Opening an RPC connection to a remote Geth node

○ Sync the entire beacon chain history

○ Store and maintain the set of active, queued and exited validators

○ Process cross-links

○ Process its own block-by-block consensus, as well as the FFG finality gadget

■ “For a block on the beacon chain to be processed by a node, three conditions have to be met:

■ The parent pointed to by the parent_hash has already been processed and accepted

■ The main chain block pointed to by the main_chain_ref has already been processed and accepted

■ The node’s local clock time is greater than or equal to the minimum timestamp as computed by GENESIS_TIME + height * PER_HEIGHT_DELAY + total_skips * PER_SKIP_DELAY

■ If these three conditions are not met, the client should delay processing the block until the three conditions are all satisfied.”

Proposers

● Proposers are responsible for signing a block and broadcasting it for the attestation committee to validate.

● Blocks include a pre-image of the proposer’s randao commitment.

● Blocks include notary signatures for shards selected during that epoch.

● Blocks include a hash of the longest PoW chain in main_chain_ref

● If a proposer fails to produce a valid block within PER_HEIGHT_DELAY, the next proposer may produce a block instead.

● The next proposer is eligible if their local time is greater than minimum_timestamp(head) + PER_HEIGHT_DELAY + skip_count * PER_SKIP_DELAY where skip_count is the number of proposers that were skipped over.

● Proposers are shuffled every block.

Notaries - Phase 1

● If assigned to a shard, notaries sign a block hash for that shard.

● If ⅔ of notaries sign a block hash, that block gets crosslinked into the random beacon chain.

● Currently, notaries per crosslink is set to 1024.

● Only a subset of shards are selected for crosslink every epoch, depending on the number of available validators.

● Notaries are shuffled once every dynasty.

Attesters

● Attesters are responsible for validating and signing blocks signed by a producer

● Blocks are valid if N/(2 + skip_count) attesters sign a block, where N is the size of the attestation committee.

● Attesters are shuffled every block.

Potential Scope Changes for Ruby Release

The following are requirements that could be removed to adjust the scope:

 

● Left out shards, just focus on beacon chain, in that case there’s no notary, just proposer and attesters. Don’t have to worry about shard linking, shard fork choice rules.

● Shard linking can be left out entirely. In this scenario notaries aren’t selected and PartialCrossLinkRecord and CrossLinkRecord are always nil.

● If Shard linking is left out, dynasty transitions can also be removed. In this scenario the validator set never changes from the initial genesis block.

● BLS signature verification can be left out. In this scenario the aggregate signature verification functions for attesters and notaries are noops that always return true.

● We can remove PoW fork choice rules and assume that the main_chain_ref is always inside the highest scoring PoW chain. In this scenario the fork choice rule simply becomes the justified checkpoint of the greatest height. Removing this requirement also allows us to not worry about a RBC reorg in the event of a PoW chain reorg.

● We can replace active_state.randao with a pseudo-random hash. In this scenario we would be able to remove the construction of the hash onion and verification of the proposer’s randao commitment. We can just use blockhashes for this.

Project Structure

 

Yutaro: The RBC should be a data structure that can not actively modify its own state. For example, instead of actively updating the validator set by polling the PoW chain via a JSON RPC, the proposer should update the RBC’s validator set via its own JSON RPC. This also means that the RBC should avoid having its own external dependencies (JSON RPC, p2p networking logic). Instead, those responsibilities should be handled by other processes. The main reason I’m arguing for this approach is that I envision state management to be handled in multiple ways, and I’d like to be able to separate the data structure from those responsibilities. Here are different ways the RBC’s state can advance:

 

● Initial sync via p2p

● New block post-sync via p2p

● Client is a validator that produces a new block

● Test setup via a static file

● Upon each change, persist to disk

 

The best way to implement these different scenarios is to separate “chain management” from “chain state”. This approach introduces another actor, “p2pSync”, that is responsible for updating the state via our p2p protocol during initial and post-initial sync. The main downside of this approach is that it introduces concurrency issues where p2pSync may write at the same time that the client updates a new block. This could be handled fairly easily by locking the RBC when updating its state. Because chain state updates happen infrequently, lock contention should not be an issue.

 

Another benefit to having a simple RBC is that integrated tests will be easier to setup without any external dependencies to mock or any unexpected state management to stub out.

Work Estimates

Needs Design

● Fork Choice Rule, Reorg

● BLS Signature Inclusion

● P2P where a validator can have multiple responsibilities

● Block transitions

● Rewards/Penalties (Slashing Conditions)

 

The following structure could be a good path forward to begin on a minimal implementation

● Setup a service registry in a similar way as we did in geth-sharding for the beacon-chain repo

● Open a JSON-RPC connection to an endpoint and listen for the latest block number, blockhash

● Have the beacon chain node listen for this event happening on the main chain, trigger a queueValidator func once the tx is mined on the main chain (listen for logs from the validator registration contract)

● Pass in the address of the validator registration contract as an arg to the beacon chain

● Beacon chain to access the private key of the validator that registered the eth in the PoW chain

● The beacon chain becomes a data structure (state machine) that is updated by external services

● Node has to sync with the canonical beacon chain (sync from the last finalized epoch)

○ Node has to run the block processing service

● Beacon node needs to sample proposer/attester responsibilities using blockhash for randomness

● Proposer running the node has to begin producing blocks via PoS

Has to apply fork choice rule

○ Compute block and epoch state transitions

● Node has to participate in a global RANDAO mechanism to select committees at respective times (Mega Issue, will have to be split into many parts)

● Logic for crosslink notaries

○ Implement getCrosslinkShards and getCrosslinkNotaries

● Logic for randao reveal from validator hash preimage

● Logic for beacon chain fork choice - design this and also reorgs

● Some web3 methods to include get_score

A New System Without Geth Moved to another doc

References ● Casper + Sharding v2