Substrate - node-template startup

entry: main()

-> cli::run

cli commands

cli commands can be found in substrate/client/cli::sc_cli

#[derive(Debug, StructOpt)]
pub enum Subcommand {
	/// Key management cli utilities
	Key(sc_cli::KeySubcommand),
	/// Build a chain specification.
	BuildSpec(sc_cli::BuildSpecCmd),

	/// Validate blocks.
	CheckBlock(sc_cli::CheckBlockCmd),

	/// Export blocks.
	ExportBlocks(sc_cli::ExportBlocksCmd),

	/// Export the state of a given block into a chain spec.
	ExportState(sc_cli::ExportStateCmd),

	/// Import blocks.
	ImportBlocks(sc_cli::ImportBlocksCmd),

	/// Remove the whole chain.
	PurgeChain(sc_cli::PurgeChainCmd),

	/// Revert the chain to a previous state.
	Revert(sc_cli::RevertCmd),

	/// The custom benchmark subcommand benchmarking runtime pallets.
	#[structopt(name = "benchmark", about = "Benchmark runtime pallets.")]
	Benchmark(frame_benchmarking_cli::BenchmarkCmd),
}

A CLI runner will be created from sc_cli::RunCmd to run the node when there is no args specified. Based on the config.role, it will be started as a full or lite node.

			let runner = cli.create_runner(&cli.run)?;
			runner.run_node_until_exit(|config| async move {
				match config.role {
					Role::Light => service::new_light(config),
					_ => service::new_full(config),
				}
				.map_err(sc_cli::Error::Service)
			})

lite node

• telemetry
• executor: NativeElseWasmExecutor
• client/backend/…
• transaction_pool
• network: sc_service::build_network

full node

new_full() to create a full node.

• new_partial: make the Client
• build_network
• rpc
• grandpa, if enabled

Client

node-template => new_partial => sc_service::new_full_parts => sc_service::new_client => Client::new

• backend: new_db_backend(db_config)
• client: new(backend, executor, genesis, task_manager, …)

pub struct Client<B, E, Block, RA>
where
	Block: BlockT,
{
	backend: Arc<B>,
	executor: E,
	storage_notifications: Mutex<StorageNotifications<Block>>,
	import_notification_sinks: NotificationSinks<BlockImportNotification<Block>>,
	finality_notification_sinks: NotificationSinks<FinalityNotification<Block>>,
	// holds the block hash currently being imported. TODO: replace this with block queue
	importing_block: RwLock<Option<Block::Hash>>,
	block_rules: BlockRules<Block>,
	execution_extensions: ExecutionExtensions<Block>,
	config: ClientConfig<Block>,
	telemetry: Option<TelemetryHandle>,
	_phantom: PhantomData<RA>,
}

Client Event

Client can emit 3 different blockchain events to the listeners:

• notify_imported() + self.storage_notifications.lock().trigger()
• notify_finalized()

The entry of all 3 events is: lock_import_and_run, and events/nofications will be sent to the event listeners via futures_channel::UnboundedSerder::unbounded_send. There might be multiple listeners for these events. A importnat one resides in the network main loop, see build_network_future.

impl<B, E, Block, RA> BlockchainEvents<Block> for Client<B, E, Block, RA>
where
	E: CallExecutor<Block>,
	Block: BlockT,
{
	/// Get block import event stream.
	fn import_notification_stream(&self) -> ImportNotifications<Block> {
		let (sink, stream) = tracing_unbounded("mpsc_import_notification_stream");
		self.import_notification_sinks.lock().push(sink);
		stream
	}

	fn finality_notification_stream(&self) -> FinalityNotifications<Block> {
		let (sink, stream) = tracing_unbounded("mpsc_finality_notification_stream");
		self.finality_notification_sinks.lock().push(sink);
		stream
	}

	/// Get storage changes event stream.
	fn storage_changes_notification_stream(
		&self,
		filter_keys: Option<&[StorageKey]>,
		child_filter_keys: Option<&[(StorageKey, Option<Vec<StorageKey>>)]>,
	) -> sp_blockchain::Result<StorageEventStream<Block::Hash>> {
		Ok(self.storage_notifications.lock().listen(filter_keys, child_filter_keys))
	}
}

Client Services

Network and RPC

There is a channel ‘system_rpc_tx/system_rpc_rx’ between two services.

Grandpa service might be started when it is enabled.

rpc

rpc_extensions_builder is created with the following APIs:

	io.extend_with(SystemApi::to_delegate(FullSystem::new(client.clone(), pool, deny_unsafe)));

	io.extend_with(TransactionPaymentApi::to_delegate(TransactionPayment::new(client.clone())));

–note-- for lite node, rpc_extensions_builder is empty

then, rpc handlers will be created:

	let _rpc_handlers = sc_service::spawn_tasks(sc_service::SpawnTasksParams {
		network: network.clone(),
		client: client.clone(),
		keystore: keystore_container.sync_keystore(),
		task_manager: &mut task_manager,
		transaction_pool: transaction_pool.clone(),
		rpc_extensions_builder,
		on_demand: None,
		remote_blockchain: None,
		backend,
		system_rpc_tx,
		config,
		telemetry: telemetry.as_mut(),
	})?;

sc_service::spawn_tasks will spawn:

• txpool-notifications
• on-transaction-imported
• prometheus-endpoint, if configured
• telemetry-periodic-send
• RpcHandler: sc_rpc_server::RpcHandler
• informant

And, RpcHandler will be created as below:

	Ok(sc_rpc_server::rpc_handler(
		(
			state::StateApi::to_delegate(state),
			state::ChildStateApi::to_delegate(child_state),
			chain::ChainApi::to_delegate(chain),
			maybe_offchain_rpc,
			author::AuthorApi::to_delegate(author),
			system::SystemApi::to_delegate(system),
			rpc_extensions_builder.build(deny_unsafe, task_executor)?,
		),
		rpc_middleware,
	))

network

substrate/client/service::sc_service

• network parameters come from sc_cli::params::NetworkParams

  • sc_cli::params::NetworkParams => NetworkConfiguration => sc_network::config::Params

• startup: sc_service::build_network && network_starter.start_network()

  • sc_service::build_network() spawn a future which consists of 3 key components:
    • Role: full/lite/authority
    • NetworkWorker: sc_network::NetworkWorker::new(network_params): initialize rust-libp2p, the Swarm/transport and protocols
    • Client: the chain itself: sc_service::TFullClient<Block, RuntimeApi, NativeElseWasmExecutor>;
    • in the end, spawn a future running a event loop to handle: imported_blocks_stream/Client, finality_notification_stream/Client, rpc_request/RPC and the Swarm itself

/// Parameters used to create the network configuration.
#[derive(Debug, StructOpt, Clone)]
pub struct NetworkParams {
	/// Specify a list of bootnodes.
	#[structopt(long = "bootnodes", value_name = "ADDR")]
	pub bootnodes: Vec<MultiaddrWithPeerId>,

	/// Specify a list of reserved node addresses.
	#[structopt(long = "reserved-nodes", value_name = "ADDR")]
	pub reserved_nodes: Vec<MultiaddrWithPeerId>,

	/// Whether to only synchronize the chain with reserved nodes.
	///
	/// Also disables automatic peer discovery.
	///
	/// TCP connections might still be established with non-reserved nodes.
	/// In particular, if you are a validator your node might still connect to other
	/// validator nodes and collator nodes regardless of whether they are defined as
	/// reserved nodes.
	#[structopt(long = "reserved-only")]
	pub reserved_only: bool,

	/// The public address that other nodes will use to connect to it.
	/// This can be used if there's a proxy in front of this node.
	#[structopt(long, value_name = "PUBLIC_ADDR")]
	pub public_addr: Vec<Multiaddr>,

	/// Listen on this multiaddress.
	///
	/// By default:
	/// If `--validator` is passed: `/ip4/0.0.0.0/tcp/<port>` and `/ip6/[::]/tcp/<port>`.
	/// Otherwise: `/ip4/0.0.0.0/tcp/<port>/ws` and `/ip6/[::]/tcp/<port>/ws`.
	#[structopt(long = "listen-addr", value_name = "LISTEN_ADDR")]
	pub listen_addr: Vec<Multiaddr>,

	/// Specify p2p protocol TCP port.
	#[structopt(long = "port", value_name = "PORT", conflicts_with_all = &[ "listen-addr" ])]
	pub port: Option<u16>,

	/// Always forbid connecting to private IPv4 addresses (as specified in
	/// [RFC1918](https://tools.ietf.org/html/rfc1918)), unless the address was passed with
	/// `--reserved-nodes` or `--bootnodes`. Enabled by default for chains marked as "live" in
	/// their chain specifications.
	#[structopt(long = "no-private-ipv4", conflicts_with_all = &["allow-private-ipv4"])]
	pub no_private_ipv4: bool,

	/// Always accept connecting to private IPv4 addresses (as specified in
	/// [RFC1918](https://tools.ietf.org/html/rfc1918)). Enabled by default for chains marked as
	/// "local" in their chain specifications, or when `--dev` is passed.
	#[structopt(long = "allow-private-ipv4", conflicts_with_all = &["no-private-ipv4"])]
	pub allow_private_ipv4: bool,

	/// Specify the number of outgoing connections we're trying to maintain.
	#[structopt(long = "out-peers", value_name = "COUNT", default_value = "25")]
	pub out_peers: u32,

	/// Specify the maximum number of incoming connections we're accepting.
	#[structopt(long = "in-peers", value_name = "COUNT", default_value = "25")]
	pub in_peers: u32,

	/// Disable mDNS discovery.
	///
	/// By default, the network will use mDNS to discover other nodes on the
	/// local network. This disables it. Automatically implied when using --dev.
	#[structopt(long = "no-mdns")]
	pub no_mdns: bool,

	/// Maximum number of peers from which to ask for the same blocks in parallel.
	///
	/// This allows downloading announced blocks from multiple peers. Decrease to save
	/// traffic and risk increased latency.
	#[structopt(long = "max-parallel-downloads", value_name = "COUNT", default_value = "5")]
	pub max_parallel_downloads: u32,

	#[allow(missing_docs)]
	#[structopt(flatten)]
	pub node_key_params: NodeKeyParams,

	/// Enable peer discovery on local networks.
	///
	/// By default this option is `true` for `--dev` or when the chain type is
	/// `Local`/`Development` and false otherwise.
	#[structopt(long)]
	pub discover_local: bool,

	/// Require iterative Kademlia DHT queries to use disjoint paths for increased resiliency in
	/// the presence of potentially adversarial nodes.
	///
	/// See the S/Kademlia paper for more information on the high level design as well as its
	/// security improvements.
	#[structopt(long)]
	pub kademlia_disjoint_query_paths: bool,

	/// Join the IPFS network and serve transactions over bitswap protocol.
	#[structopt(long)]
	pub ipfs_server: bool,

	/// Blockchain syncing mode.
	///
	/// - `Full`: Download and validate full blockchain history.
	///
	/// - `Fast`: Download blocks and the latest state only.
	///
	/// - `FastUnsafe`: Same as `Fast`, but skip downloading state proofs.
	#[structopt(long, value_name = "SYNC_MODE", default_value = "Full")]
	pub sync: SyncMode,
}