ipfs, libp2p stream peformance test

Test ENV

Two libp2p hosts running on the same PC, communicating to each other with loopback interface

Host A -> Host B, uni-direction

Case 1, secio, AES+SHA

19:19:34.257 ERROR     common: Copied=1000M, rate=51.39MB/s, elapsed=19.458s common.go:1001
19:19:55.198 ERROR     common: Copied=1000M, rate=57.77MB/s, elapsed=17.309s common.go:1001
19:20:14.228 ERROR     common: Copied=1000M, rate=57.98MB/s, elapsed=17.247s common.go:1001

Case 2, tls, AES-GCM-128 by default

19:16:16.947 ERROR     common: Copied=1000M, rate=80.40MB/s, elapsed=12.438s common.go:1002
19:16:32.845 ERROR     common: Copied=1000M, rate=79.87MB/s, elapsed=12.521s common.go:1002
19:16:50.199 ERROR     common: Copied=1000M, rate=78.85MB/s, elapsed=12.682s common.go:1002

Case 3, no security

19:21:53.118 ERROR     common: Copied=1000M, rate=101.13MB/s, elapsed=9.888s common.go:1002
19:22:06.980 ERROR     common: Copied=1000M, rate=106.45MB/s, elapsed=9.394s common.go:1002
19:22:18.085 ERROR     common: Copied=1000M, rate=98.71MB/s, elapsed=10.131s common.go:1002

Case 4, Private Network(salsa20) + no security

21:22:14.351 ERROR     common: Copied=1000M, rate=79.69MB/s, elapsed=12.549s common.go:1031
21:22:32.460 ERROR     common: Copied=1000M, rate=80.20MB/s, elapsed=12.469s common.go:1031
21:22:46.370 ERROR     common: Copied=1000M, rate=79.90MB/s, elapsed=12.516s common.go:1031

Case 5, Private Network(salsa20) + secio

21:25:20.243 ERROR     common: Copied=1000M, rate=50.35MB/s, elapsed=19.86s common.go:1031
21:25:43.967 ERROR     common: Copied=1000M, rate=50.66MB/s, elapsed=19.738s common.go:1031
21:26:05.711 ERROR     common: Copied=1000M, rate=52.85MB/s, elapsed=18.923s common.go:1031

Case 6, no libp2p, TCP only

10:24:24.036 ERROR     common: Copied=1000M, rate=143.54MB/s, elapsed=6.9668s common.go:1137
10:24:48.280 ERROR     common: Copied=1000M, rate=147.85MB/s, elapsed=6.7638s common.go:1137
10:24:56.844 ERROR     common: Copied=1000M, rate=145.98MB/s, elapsed=6.8504s common.go:1137

Conclusion

• secio is quite slow, need GCM support in future
• salsa20 is good, but probably less secure than AES
• libp2p comes with overhead

New findings

As yamux.Write() can do fragmentation, it’s better to stream.Write() more bytes as possible when trying to send out lots of data. This can save some time by reducing multiple stream.Write invokations. A good idea is to use bufio.reader/writer with bigger BufSize, NewWriterSize(s, 32768) …

Case 7, secio with AES-GCM-128

Made some changes to secio, added AES-GCM support, then run the test again

Copied=1000M, rate=78.37MB/s, elapsed=12.76s
Copied=1000M, rate=87.23MB/s, elapsed=11.464s
Copied=1000M, rate=85.10MB/s, elapsed=11.751s

We can see the performance benchmark is simliar with the test case 2 tls.