bitswap wantmanager代码解读

代码版本：go-bitswap@v0.1.11

wantlist

目前bitswap代码有两处用到了wantlist，一处是engine，另外一处是wantmanager。

engine.ledgerMap中的wantlist是Wantlist类型，记录的是别的peer的wantlist。

wantmanager.wl用于记录本节点所有session的wantlist， wantmanager.bcwl用于记录本节点所有session广播的wantlist。
两者都是SessionTrackedWantlist类型的，相对Entry，sessionTrackedEntry多了一个sesTrk，用于记录那些session需要该块（key是session ID）。

// SessionTrackedWantlist is a list of wants that also track which bitswap
// sessions have requested them
type SessionTrackedWantlist struct {
	set map[cid.Cid]*sessionTrackedEntry
}

// Wantlist is a raw list of wanted blocks and their priorities
type Wantlist struct {
	set map[cid.Cid]Entry
}

// Entry is an entry in a want list, consisting of a cid and its priority
type Entry struct {
	Cid      cid.Cid
	Priority int
}

type sessionTrackedEntry struct {
	Entry
	sesTrk map[uint64]struct{}
}

本文主要集中在对wantmanager的分析。

WantManager结构

// WantManager manages a global want list. It tracks two seperate want lists -
// one for all wants, and one for wants that are specifically broadcast to the
// internet.
type WantManager struct {
	// channel requests to the run loop
	// to get predictable behavior while running this in a go routine
	// having only one channel is neccesary, so requests are processed serially
	wantMessages chan wantMessage

	// synchronized by Run loop, only touch inside there
	wl   *wantlist.SessionTrackedWantlist
	bcwl *wantlist.SessionTrackedWantlist

	ctx    context.Context
	cancel func()

	peerHandler   PeerHandler
	wantlistGauge metrics.Gauge
}

WantManager主要包含wantMessages、wl、bcwl和peerHandler。

wantMessage

wantMessage有6种消息：

wantSet
currentWantsMessage
currentBroadcastWantsMessage
wantCountMessage
connectedMessage
disconnectedMessage

其中，wantSet用于WantBlocks()和CancelWants()，这2个函数是session调用WantManager的主要接口;

currentWantsMessage和currentBroadcastWantsMessage用于获取当前的wantlist和Broadcast WantList，它们分别记录在WantManager.wl和WantManager.bcwl;

wantCountMessage用于获取当前的wantlist的大小；

connectedMessage用于通知WantManager有新的连接，wantmanager会首先把新的peer保存到wantmanager.peerHandler.peerQueues，如果已经存在，引用计数(refcnt)+1，然后向新的peer询问是否有正在brocast的wantlists（记录在bs.wantmanager.bcwl）;

disconnectedMessage用于通知wantmanager某连接断开了，wantmanager会首先wantmanager.peerHandler.peerQueues.refcnt-1，然后判断refcnt是否为0，如果为0 ，delete(peerQueues, p);

wl, bcwl

wl用于记录本节点所有session的wantlist， bcwl用于记录本节点所有session广播的wantlist。

wl和bcwl都是SessionTrackedWantlist类型的，key是block CID, value是sessionTrackedEntry，前面已经提到，这里不再赘述。

// SessionTrackedWantlist is a list of wants that also track which bitswap
// sessions have requested them
type SessionTrackedWantlist struct {
	set map[cid.Cid]*sessionTrackedEntry
}

type sessionTrackedEntry struct {
	Entry
	sesTrk map[uint64]struct{}
}

peerHandler

wantmanager通过peerHandler(peermanager类型)为每个连接的peer维护一个messagequeue，messagequeue保存着sender和SessionTrackedWantlist，sender主要记录发送所用的流，sendMessage()时如果该sender不存在，则先创建连接和流。

wantmanager.peerHandler维护的peerQueues的来源有2个：

1、新连接: 当有新的连接时，libp2p swarm会通知bitswap，bitswap会通知wantmanager。比如在findprovider的过程中connect的closer peers后也会通知bitswap；

2、session通过dht找到providers，调用wantmanager.WantBlocks()将peer添加到peerQueue中。

type wantSet struct {
	entries []bsmsg.Entry
	targets []peer.ID
	from    uint64
}

func (ws *wantSet) handle(wm *WantManager) {
	// is this a broadcast or not?
	brdc := len(ws.targets) == 0

	// add changes to our wantlist
	for _, e := range ws.entries {
		if e.Cancel {
			if brdc {
				wm.bcwl.Remove(e.Cid, ws.from)
			}

			if wm.wl.Remove(e.Cid, ws.from) {
				wm.wantlistGauge.Dec()
			}
		} else {
			if brdc {
				wm.bcwl.AddEntry(e.Entry, ws.from)
			}
			if wm.wl.AddEntry(e.Entry, ws.from) {
				wm.wantlistGauge.Inc()
			}
		}
	}

	// broadcast those wantlist changes
	wm.peerHandler.SendMessage(ws.entries, ws.targets, ws.from)
}

type connectedMessage struct {
	p peer.ID
}

func (cm *connectedMessage) handle(wm *WantManager) {
	wm.peerHandler.Connected(cm.p, wm.bcwl)
}

type disconnectedMessage struct {
	p peer.ID
}

func (dm *disconnectedMessage) handle(wm *WantManager) {
	wm.peerHandler.Disconnected(dm.p)
}

peerManage

peerQueues为每个连接的peer维护一个messagequeue，

type peerQueueInstance struct {
	refcnt int
	pq     PeerQueue
}

// PeerManager manages a pool of peers and sends messages to peers in the pool.
type PeerManager struct {
	// peerQueues -- interact through internal utility functions get/set/remove/iterate
	peerQueues map[peer.ID]*peerQueueInstance

	createPeerQueue PeerQueueFactory
	ctx             context.Context
}

messagequeue

p记录连接的peerID，sender主要记录发送所用的流，wl记录对该peer的SessionTrackedWantlist。

// MessageQueue implements queue of want messages to send to peers.
type MessageQueue struct {
	ctx     context.Context
	p       peer.ID
	network MessageNetwork

	outgoingWork chan struct{}
	done         chan struct{}

	// do not touch out of run loop
	wl                    *wantlist.SessionTrackedWantlist
	nextMessage           bsmsg.BitSwapMessage
	nextMessageLk         sync.RWMutex
	sender                bsnet.MessageSender
	rebroadcastIntervalLk sync.RWMutex
	rebroadcastInterval   time.Duration
	rebroadcastTimer      *time.Timer
}

MessageSender

// ipfs_impl.go
type streamMessageSender struct {
	s     network.Stream
	bsnet *impl
}

func (s *streamMessageSender) Close() error {
	return helpers.FullClose(s.s)
}

func (s *streamMessageSender) Reset() error {
	return s.s.Reset()
}

func (s *streamMessageSender) SendMsg(ctx context.Context, msg bsmsg.BitSwapMessage) error {
	return s.bsnet.msgToStream(ctx, s.s, msg)
}

func (bsnet *impl) NewMessageSender(ctx context.Context, p peer.ID) (MessageSender, error) {
	s, err := bsnet.newStreamToPeer(ctx, p)
	if err != nil {
		return nil, err
	}

	return &streamMessageSender{s: s, bsnet: bsnet}, nil
}

如果该sender不存在，则调用Connect()创建连接，调用NewMessageSender()创建流。

func openSender(ctx context.Context, network MessageNetwork, p peer.ID) (bsnet.MessageSender, error) {
	// allow ten minutes for connections this includes looking them up in the
	// dht dialing them, and handshaking
	conctx, cancel := context.WithTimeout(ctx, time.Minute*10)
	defer cancel()

	err := network.ConnectTo(conctx, p)
	if err != nil {
		return nil, err
	}

	nsender, err := network.NewMessageSender(ctx, p)
	if err != nil {
		return nil, err
	}

	return nsender, nil
}

如果sender存在，调用SendMsg()。SendMsg()最终将消息序列化(proto)后，写到流里面。

func (mq *MessageQueue) attemptSendAndRecovery(message bsmsg.BitSwapMessage) bool {
	err := mq.sender.SendMsg(mq.ctx, message)
	if err == nil {
		return true
	}

	...
}

BitSwapMessage

虽然message中包含wantlist和blocks，也支持同时解析block和wantlist，但是两者目前是分离的，即一条消息只有wantlist或者block。

tips:

对于other peer’s wantlist， 由engine决定是否发送，最终通过bs.taskWorker()调用sendBlocks()发送出去; 对于本节点的wantlist，由wantmanager管理，最终通过messagequeue.attemptSendAndRecovery()调用SendMsg()发送出去。

对于block，发送前NewStream，发送完close；对于wantlist，messagequeue会维护已创建的stream，方便以后复用。

type impl struct {
	full     bool
	wantlist map[cid.Cid]*Entry
	blocks   map[cid.Cid]blocks.Block
}

总结

执行ipfs get命令时, ipfs首先会查找本地blockstore是否有对应的块，如果没有该块，调用NewSession，由session负责查找。

session会调用wantmanager.WantBlocks()，将wantlist添加到wantmanager.wl，wantmanager通过peerHandler(peermanager类型)为每个连接的peer维护一个messagequeue，messagequeue保存着向该peer发送的SessionTrackedWantlist，SessionTrackedWantlist的key是block CID, value是sessionTrackedEntry， session ID（作为key）保存在sessionTrackedEntry.sesTrk。

session如果收到了block，会调用wantmanager.CancelWants()，从而 delete(SessionTrackedWantlist, session ID)， 表明该session不想要该block了。

wantmanager是在new bitswap的时候被实例化，并且代入bitswap.wm，以后newsession时作为参数传入到session中。也就是说wantmanager管理的是本节点所有session的wantlist。

值得注意的是，wantmanager是被动的，机械的，只有WantBlocks()、CancelWants()以及Connected()、Disconnected()被调用才会运转，前三者都会向与之连接的peer发送wantlist请求。