swarm, conn & stream
In Swarm, conn stands for the connection, on which streams are multiplexed.
First of all, we must create a connection before starting any stream operations. A connection can be made by swarm.dial() or swarm.listen(), outbound or inbound
type Conn struct {
conn transport.CapableConn
swarm *Swarm
closeOnce sync.Once
err error
notifyLk sync.Mutex
streams struct {
sync.Mutex
m map[*Stream]struct{}
}
stat network.Stat
}
swarm.addConn
Whenever swarm get a incoming/outgoing connection, addConn will be called. To be more specific, swarm.dial() or swarm.listen()
notifyAll() for Connetced notification
start() a routine for stream handling
go s.ConnHandler() to invoke connHandler registered in Swarm
...
s.notifyAll(func(f network.Notifiee) {
f.Connected(s, c)
})
c.notifyLk.Unlock()
c.start()
// TODO: Get rid of this. We use it for identify but that happen much
// earlier (really, inside the transport and, if not then, during the
// notifications).
if h := s.ConnHandler(); h != nil {
go h(c)
}
return c, nil
}
go h(c) —> our conn handler will be executed in a new routine
Take a look at start() : It will listen to new stream comming and addStream or so.
func (c *Conn) start() {
go func() {
defer c.swarm.refs.Done()
defer c.Close()
for {
ts, err := c.conn.AcceptStream()
if err != nil {
log.Error(err, c)
return
}
c.swarm.refs.Add(1)
go func() {
s, err := c.addStream(ts, network.DirInbound)
// Don't defer this. We don't want to block
// swarm shutdown on the connection handler.
c.swarm.refs.Done()
// We only get an error here when the swarm is closed or closing.
if err != nil {
return
}
if h := c.swarm.StreamHandler(); h != nil {
h(s)
}
}()
}
}()
}
AcceptStream() of yamux: Waiting for new imcoming stream on channel
func (s *Session) AcceptStream() (*Stream, error) {
select {
case stream := <-s.acceptCh:
if err := stream.sendWindowUpdate(); err != nil {
return nil, err
}
return stream, nil
case <-s.shutdownCh:
return nil, s.shutdownErr
}
}
swarm.connHandler
Only got a new connection, start IdentifyService asap
IdentifyConn will create a new stream, exchange protocol ID and try to get Identify from peer
// newConnHandler is the remote-opened conn handler for inet.Network
func (h *BasicHost) newConnHandler(c network.Conn) {
// Clear protocols on connecting to new peer to avoid issues caused
// by misremembering protocols between reconnects
h.Peerstore().SetProtocols(c.RemotePeer())
h.ids.IdentifyConn(c)
}
swarm.NewStream Inbound
Incoming msg will be handled by underlying yamux. Msg with flagSYN will be treated as a new imcoming stream. yamux creates a new stream and pass it to a channel (acceptCh), which is listened/read by yamux.AcceptStream(). See above.
swarm.NewStream Outbound
IdentifyConn as example, call stack:
github.com/libp2p/go-yamux.(*Session).OpenStream at session.go:194
github.com/libp2p/go-libp2p-yamux.(*conn).OpenStream at yamux.go:28
<autogenerated>:2
github.com/libp2p/go-libp2p-swarm.(*Conn).NewStream at swarm_conn.go:172
github.com/libp2p/go-libp2p/p2p/protocol/identify.(*IDService).IdentifyConn at id.go:185
github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).newConnHandler at basic_host.go:247
github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).newConnHandler-fm at basic_host.go:243
runtime.goexit at asm_amd64.s:1357
- Async stack trace
github.com/libp2p/go-libp2p-swarm.(*Swarm).addConn at swarm.go:243
- NewStream -> mux.SelectOneOf(ProtocolID) -> SelectProtoOrFail: Neogotiate the protocol
- Write("/multistream/1.0.0") -> Write(Proto)
- Read("/multistream/1.0.0") -> Read(Proto)
func SelectProtoOrFail(proto string, rwc io.ReadWriteCloser) error {
errCh := make(chan error, 1)
go func() {
var buf bytes.Buffer
delimWrite(&buf, []byte(ProtocolID))
delimWrite(&buf, []byte(proto))
_, err := io.Copy(rwc, &buf)
errCh <- err
}()
// We have to read *both* errors.
err1 := readMultistreamHeader(rwc)
err2 := readProto(proto, rwc)
if werr := <-errCh; werr != nil {
return werr
}
if err1 != nil {
return err1
}
if err2 != nil {
return err2
}
return nil
}
swarm.StreamHandler
mux.Negotiate will exchange protocl ID and determine which handler to use by looking up the handler map
func (h *BasicHost) newStreamHandler(s network.Stream) {
before := time.Now()
if h.negtimeout > 0 {
if err := s.SetDeadline(time.Now().Add(h.negtimeout)); err != nil {
log.Error("setting stream deadline: ", err)
s.Reset()
return
}
}
lzc, protoID, handle, err := h.Mux().NegotiateLazy(s)
took := time.Since(before)
if err != nil {
if err == io.EOF {
logf := log.Debugf
if took > time.Second*10 {
logf = log.Warningf
}
logf("protocol EOF: %s (took %s)", s.Conn().RemotePeer(), took)
} else {
log.Debugf("protocol mux failed: %s (took %s)", err, took)
}
s.Reset()
return
}
s = &streamWrapper{
Stream: s,
rw: lzc,
}
if h.negtimeout > 0 {
if err := s.SetDeadline(time.Time{}); err != nil {
log.Error("resetting stream deadline: ", err)
s.Reset()
return
}
}
s.SetProtocol(protocol.ID(protoID))
log.Debugf("protocol negotiation took %s", took)
go handle(protoID, s)
}
NegotiateLazy() —> determine the protocol and such
go handle(protoID, s) —> our stream handler will be executed in a new routine
stream.Write
stream.Write -> multistream.Write -> yamux.Write -> fragment(max 65535) if need ->session.sendMsg
Call Stack
github.com/libp2p/go-yamux.(*Session).sendMsg at session.go:358
github.com/libp2p/go-yamux.(*Stream).write at stream.go:172
github.com/libp2p/go-yamux.(*Stream).Write at stream.go:128
github.com/libp2p/go-libp2p-swarm.(*Stream).Write at swarm_stream.go:88
github.com/multiformats/go-multistream.(*lazyServerConn).Write at lazyServer.go:25
github.com/libp2p/go-libp2p/p2p/host/basic.(*streamWrapper).Write at basic_host.go:804
main.serveStub.func1.1 at proxy.go:45
runtime.goexit at asm_amd64.s:1357
- Async stack trace
main.serveStub.func1 at proxy.go:31
yamux will firstly fragment the msg body if necessary then do the stream header encoding and finally send msg to the sendloop routine through a channel
yamux creates a sendLoop routine waiting for buf sent from Mux session, when creating the yamux session
Details can be found in yamux in details