filecoin-lotus链同步、链本地持久化存储源码分析

一、链存储

源码位置：
官方文档 documentation/en/architecture/architecture.md
chain/store/store.go

Store的设计基本是给ChainSync使用的

链存储 相关类型 ChainStore，

ChainStore的成员变量 ds dstore.Datastore它key-value形式，比如key 是 “head”表示链头，value就是 cids字符串数组的json.marshal操作。我们通过其的Get Put方法读写链头信息。

// ChainStore is the main point of access to chain data.
//
// Raw chain data is stored in the Blockstore, with relevant markers (genesis,
// latest head tipset references) being tracked in the Datastore (key-value
// store).
//
// To alleviate disk access, the ChainStore has two ARC caches:
//   1. a tipset cache
//   2. a block => messages references cache.
type ChainStore struct {
	bs bstore.Blockstore
	ds dstore.Datastore

	heaviestLk sync.Mutex
	heaviest   *types.TipSet

	bestTips *pubsub.PubSub
	pubLk    sync.Mutex

	tstLk   sync.Mutex
	tipsets map[abi.ChainEpoch][]cid.Cid

	cindex *ChainIndex

	reorgCh        chan<- reorg
	reorgNotifeeCh chan ReorgNotifee

	mmCache *lru.ARCCache
	tsCache *lru.ARCCache

	vmcalls vm.SyscallBuilder

	evtTypes [1]journal.EventType
}

最终调Write 接口的 Put 方法写入，数据

/*
Datastore represents storage for any key-value pair.

Datastores are general enough to be backed by all kinds of different storage:
in-memory caches, databases, a remote datastore, flat files on disk, etc.

The general idea is to wrap a more complicated storage facility in a simple,
uniform interface, keeping the freedom of using the right tools for the job.
In particular, a Datastore can aggregate other datastores in interesting ways,
like sharded (to distribute load) or tiered access (caches before databases).

While Datastores should be written general enough to accept all sorts of
values, some implementations will undoubtedly have to be specific (e.g. SQL
databases where fields should be decomposed into columns), particularly to
support queries efficiently. Moreover, certain datastores may enforce certain
types of values (e.g. requiring an io.Reader, a specific struct, etc) or
serialization formats (JSON, Protobufs, etc).

IMPORTANT: No Datastore should ever Panic! This is a cross-module interface,
and thus it should behave predictably and handle exceptional conditions with
proper error reporting. Thus, all Datastore calls may return errors, which
should be checked by callers.
*/
type Datastore interface {
	Read
	Write
	// Sync guarantees that any Put or Delete calls under prefix that returned
	// before Sync(prefix) was called will be observed after Sync(prefix)
	// returns, even if the program crashes. If Put/Delete operations already
	// satisfy these requirements then Sync may be a no-op.
	//
	// If the prefix fails to Sync this method returns an error.
	Sync(prefix Key) error
	io.Closer
}

// Write is the write-side of the Datastore interface.
type Write interface {
	// Put stores the object `value` named by `key`.
	//
	// The generalized Datastore interface does not impose a value type,
	// allowing various datastore middleware implementations (which do not
	// handle the values directly) to be composed together.
	//
	// Ultimately, the lowest-level datastore will need to do some value checking
	// or risk getting incorrect values. It may also be useful to expose a more
	// type-safe interface to your application, and do the checking up-front.
	Put(key Key, value []byte) error

	// Delete removes the value for given `key`. If the key is not in the
	// datastore, this method returns no error.
	Delete(key Key) error
}

// Read is the read-side of the Datastore interface.
type Read interface {
	// Get retrieves the object `value` named by `key`.
	// Get will return ErrNotFound if the key is not mapped to a value.
	Get(key Key) (value []byte, err error)

	// Has returns whether the `key` is mapped to a `value`.
	// In some contexts, it may be much cheaper only to check for existence of
	// a value, rather than retrieving the value itself. (e.g. HTTP HEAD).
	// The default implementation is found in `GetBackedHas`.
	Has(key Key) (exists bool, err error)

	// GetSize returns the size of the `value` named by `key`.
	// In some contexts, it may be much cheaper to only get the size of the
	// value rather than retrieving the value itself.
	GetSize(key Key) (size int, err error)

	// Query searches the datastore and returns a query result. This function
	// may return before the query actually runs. To wait for the query:
	//
	//   result, _ := ds.Query(q)
	//
	//   // use the channel interface; result may come in at different times
	//   for entry := range result.Next() { ... }
	//
	//   // or wait for the query to be completely done
	//   entries, _ := result.Rest()
	//   for entry := range entries { ... }
	//
	Query(q query.Query) (query.Results, error)
}

调写链头的方法，如下，就像我们平时使用redis一样，一个key，一个value。跟踪代码这里的 key就是字符串 “head”，value就是 data, err := json.Marshal(ts.Cids()) cid数组进行 json.Marshal运算！

func (cs *ChainStore) writeHead(ts *types.TipSet) error {
	data, err := json.Marshal(ts.Cids())
	if err != nil {
		return xerrors.Errorf("failed to marshal tipset: %w", err)
	}

	if err := cs.ds.Put(chainHeadKey, data); err != nil {
		return xerrors.Errorf("failed to write chain head to datastore: %w", err)
	}

	return nil
}

最终保存在 .sst 后缀结尾的文件中。

1. lotus启动时, 从本地持久化存储的链头开始同步

如果你调（lotus wallet delete t3sg3kv6bhhor2u3op5p66pjafecckz5e5reybz23adoe7x2kgrdt4d7dmgqeaeudphh4qvbspjeab3ngedexa）删除链头的区块，启动就会报错如下：

2020-10-02T19:57:51.132+0800	WARN	modules	modules/chain.go:95	loading chain state from disk: loading tipset: get block bafy2bzacecuqqaetih7dh7o64vupkmglkcht4anunxkuuecyulyhqwygcebnm: blockstore: block not found

链头的区块找不到，就会启动不起来，报错。

跟代码,

func ChainStore(lc fx.Lifecycle, bs dtypes.ChainBlockstore, ds dtypes.MetadataDS, syscalls vm.SyscallBuilder) *store.ChainStore {
	chain := store.NewChainStore(bs, ds, syscalls)

	if err := chain.Load(); err != nil {
		log.Warnf("loading chain state from disk: %s", err)
	}

	return chain
}

chain.Load(); 是从你本地持久化链加载链。

继续跟Load()方法

func (cs *ChainStore) Load() error {
	head, err := cs.ds.Get(chainHeadKey)
	if err == dstore.ErrNotFound {
		log.Warn("no previous chain state found")
		return nil
	}
	if err != nil {
		return xerrors.Errorf("failed to load chain state from datastore: %w", err)
	}

	var tscids []cid.Cid
	if err := json.Unmarshal(head, &tscids); err != nil {
		return xerrors.Errorf("failed to unmarshal stored chain head: %w", err)
	}

	ts, err := cs.LoadTipSet(types.NewTipSetKey(tscids...))
	if err != nil {
		return xerrors.Errorf("loading tipset: %w", err)
	}

	cs.heaviest = ts

	return nil
}

它是从ChainStore的成员变量ds的Get方法，传 “head” 这个key，那对链头对应 的cid数组。通过cid数组 获取TipSet对象（ts, err := cs.LoadTipSet(types.NewTipSetKey(tscids…))）；
然后ChainStore设置成员变量heaviest为 刚才获取的TipSet对象。

二、链相关常用命令

1. 设置从某个高度开始同步

重新设置本地链头，从指定高度重新同步

lotus chain sethead --epoch 88888 bafy2bzacedehz4yobna5rcs4ozionwjxsdmj7svlfu4aqgx74mqf7gfcu6tb6

代码搜索关键字： sethead

2. 链导出

命令：

lotus chain export xxx.car

源码路径：chain\store\store.go

如下，似乎可以指定某个高度的tipset 进行导出（未测试）

var exportChainCmd = &cli.Command{
	Name:        "export",
	Description: "Export chain from repo (requires node to be offline)",
	Flags: []cli.Flag{
		&cli.StringFlag{
			Name:  "repo",
			Value: "~/.lotus",
		},
		&cli.StringFlag{
			Name:  "tipset",
			Usage: "tipset to export from",
		},
		&cli.Int64Flag{
			Name: "recent-stateroots",
		},
		&cli.BoolFlag{
			Name: "full-state",
		},
		&cli.BoolFlag{
			Name: "skip-old-msgs",
		},
	},

三、参考

filecoin技术架构分析之十四：filecoin源码分析之服务层链同步、共识协议及挖矿
参考URL: https://blog.csdn.net/qq_21393091/article/details/88373748