2014-twitter-meetup-netty

1. No Pipelining Optimization

public class HttpHandler extends SimpleChannelInboundHandler<HttpRequest> {

  @Override

  public void channelRead(ChannelHandlerContext ctx, HttpRequest req) {

    ChannelFuture future = ctx.writeAndFlush(createResponse(req)); //Write to the Channel and flush out to the Socket.

    if (!isKeepAlive(req)) {

      future.addListener(ChannelFutureListener.CLOSE); //After written close Socket

    }

  }

}

2.Pipelining to safe syscalls!

public class HttpPipeliningHandler extends SimpleChannelInboundHandler<HttpRequest> {

  @Override

  public void channelRead(ChannelHandlerContext ctx, HttpRequest req) {

    ChannelFuture future = ctx.write(createResponse(req)); //Write to the Channel (No syscall!) but don’t flush yet

    if (!isKeepAlive(req)) {

      future.addListener(ChannelFutureListener.CLOSE); //Close socket when done writing

    }

  }

  @Override

  public void channelReadComplete(ChannelHandlerContext ctx) {

    ctx.flush(); //    Flush out to the socket.

  }

}

3.Validate headers or not ?

--Validate headers for US-ASCII

ChannelPipeline pipeline = channel.pipeline();

pipeline.addLast(new HttpRequestDecoder(4096, 8192, 8192));

HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpStatus.OK);

--Not validate headers for US-ASCII

ChannelPipeline pipeline = channel.pipeline();

pipeline.addLast(new HttpRequestDecoder(4096, 8192, 8192, false));

HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpStatus.OK, false);

--Validation takes time and most of the times it is not needed directly in the decoder/encoder.

4.Static header names and values via HttpHeaders.newEntity(…).

private static final CharSequence X_HEADER_NAME = HttpHeaders.newEntity("X-Header"); //Create CharSequence for often used header names and values.

private static final CharSequence X_VALUE = HttpHeaders.newEntity("Value");

pipeline.addLast(new SimpleChannelInboundHandler<FullHttpRequest>() {

  @Override

  public void channelRead(ChannelHandlerContext ctx, FullHttpRequest req) {

    FullHttpResponse response = new FullHttpResponse(HTTP_1_1, OK);

    response.headers().set(X_HEADER_NAME, X_VALUE); //Add to HttpHeader of FullHttpResponse

    ...

    ctx.writeAndFlush(response);

  }

});

--Created CharSequence is faster to encode and faster to find in HttpHeaders.

5.write(msg) , flush() and writeAndFlush(msg)

write(msg) ⇒ pass through pipeline

flush() ⇒ gathering write of previous written msgs

writeAndFlush() ⇒ short-cut for write(msg) and flush()

--Limit flushes as much as possible as syscalls are quite expensive.

--But also limit write(...) as much as possible as it need to traverse the whole pipeline.

6.write(msg) , flush() and writeAndFlush(msg)

write(msg) ⇒ pass through pipeline

flush() ⇒ gathering write of previous written msgs

writeAndFlush() ⇒ short-cut for write(msg) and flush()

--Limit flushes as much as possible as syscalls are quite expensive.

--But also limit write(...) as much as possible as it need to traverse the whole pipeline.

7.May write too fast!

public class BlindlyWriteHandler extends ChannelInboundHandlerAdapter {

  @Override

  public void channelActive(ChannelHandlerContext ctx) throws Exception {

    while(needsToWrite) {

        ctx.writeAndFlush(createMessage());

    }

  }

}

--Writes till needsToWrite returns false.

--Risk of OutOfMemoryError if writing too fast and having slow receiver!

8.Correctly write with respect to slow receivers

public class GracefulWriteHandler extends ChannelInboundHandlerAdapter {

  @Override

  public void channelActive(ChannelHandlerContext ctx) {

    writeIfPossible(ctx.channel());

  }

  @Override

  public void channelWritabilityChanged(ChannelHandlerContext ctx) {

    writeIfPossible(ctx.channel());

  }

  private void writeIfPossible(Channel channel) {

    while(needsToWrite && channel.isWritable()) { //Make proper use of Channel.isWritable() to prevent OutOfMemoryError

      channel.writeAndFlush(createMessage());

    }

  }

}

9.Configure high and low write watermarks

Set sane WRITE_BUFFER_HIGH_WATER_MARK and WRITE_BUFFER_LOW_WATER_MARK

Server

ServerBootstrap bootstrap = new ServerBootstrap();

bootstrap.childOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 32 * 1024);

bootstrap.childOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, 8 * 1024);

Client

Bootstrap bootstrap = new Bootstrap();

bootstrap.option(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 32 * 1024);

bootstrap.option(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, 8 * 1024);

10.Issues with using non pooled buffers

--Use unpooled buffers with caution!

​-- Allocation / Deallocation is slow

​-- Free up direct buffers == PITA!

--Use pooled buffers!

Bootstrap bootstrap = new Bootstrap();

bootstrap.option(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT);

11.Use Pooling of buffers to reduce allocation / deallocation time!

12.Always use direct ByteBuffer when writing to SocketChannel

OpenJDK and Oracle JDK copy otherwise to direct buffer by itself!

Only use heap buffers if need to operate on byte[]` in ChannelOutboundHandler! By default direct ByteBuf` will be returned by ByteBufAllocator.buffer(...).

Take this as rule of thumb

13.Find pattern in ByteBuf

SlowSearch :(

ByteBuf buf = ...;

int index = -1;

for (int i = buf.readerIndex(); index == -1 && i <  buf.writerIndex(); i++) {

  if (buf.getByte(i) == '\n') {

    index = i;

  }

}

FastSearch :)

ByteBuf buf = ...;

int index = buf.forEachByte(new ByteBufProcessor() {

  @Override

  public boolean process(byte value) {

    return value != '\n';

  }

});

14.Messages with Payload? Yes please…

ByteBuf payload ⇒ extend DefaultByteBufHolder

​ --reference-counting for free

​ --release resources out-of-the-box

15.File transfer ?

--Use zero-memory-copy for efficient transfer of raw file content

Channel channel = ...;

FileChannel fc = ...;

channel.writeAndFlush(new DefaultFileRegion(fc, 0, fileLength));

--This only works if you don’t need to modify the data on the fly. If so use ChunkedWriteHandler and NioChunkedFile.

16.Never block the EventLoop!

--Thread.sleep()

--CountDownLatch.await() or any other blocking operation from java.util.concurrent

--Long-lived computationally intensive operations

--Blocking operations that might take a while (e.g. DB query)

17.Re-use EventLoopGroup if you can!

Bootstrap bootstrap = new Bootstrap().group(new NioEventLoopGroup());

Bootstrap bootstrap2 = new Bootstrap().group(new NioEventLoopGroup());

Share EventLoopGroup between different Bootstraps

EventLoopGroup group = new NioEventLoopGroup();

Bootstrap bootstrap = new Bootstrap().group(group);

Bootstrap bootstrap2 = new Bootstrap().group(group);

--Sharing the same EventLoopGroup allows to keep the resource usage (like Thread-usage) to a minimum.

18.Proxy like application with context-switching issue

public class ProxyHandler extends ChannelInboundHandlerAdapter {

  @Override

  public void channelActive(ChannelHandlerContext ctx) { //Called once a new connection was accepted

    final Channel inboundChannel = ctx.channel();

    Bootstrap b = new Bootstrap();

    b.group(new NioEventLooopGroup()); Use a new EventLoopGroup instance to handle the connection to the remote peer/

    ...

    ChannelFuture f = b.connect(remoteHost, remotePort);

    ...

  }

}

Don’t do this! This will tie up more resources than needed and introduce extra context-switching overhead.

19.Proxy like application which reduce context-switching to minimum

public class ProxyHandler extends ChannelInboundHandlerAdapter {

  @Override

  public void channelActive(ChannelHandlerContext ctx) { //Called once a new connection was accepted

    final Channel inboundChannel = ctx.channel();

    Bootstrap b = new Bootstrap();

    b.group(inboundChannel.eventLoop()); //Share the same EventLoop between both Channels. This means all IO for both connected Channels are handled by the same Thread.

    ...

    ChannelFuture f = b.connect(remoteHost, remotePort);

    ...

  }

}

Always share EventLoop in those Applications

20.Operations from inside ChannelHandler

public class YourHandler extends ChannelInboundHandlerAdapter {

  @Override

  public void channelActive(ChannelHandlerContext ctx) {

    // BAD (most of the times)

    ctx.channel().writeAndFlush(msg); //Channel.* methods ⇒ the operation will start at the tail of the ChannelPipeline

    // GOOD

    ctx.writeAndFlush(msg); //ChannelHandlerContext.* methods => the operation will start from this `ChannelHandler to flow through the ChannelPipeline.

   }

}

Use the shortest path as possible to get the maximal performance.

21.Share ChannelHandlers if stateless

@ChannelHandler.Shareable //Annotate ChannelHandler that are stateless with @ChannelHandler.Shareable and use the same instance accross Channels to reduce GC.

public class StatelessHandler extends ChannelInboundHandlerAdapter {

  @Override

  public void channelActive(ChannelHandlerContext ctx) {

    logger.debug("Now client from " + ctx.channel().remoteAddress().toString());

   }

}

public class MyInitializer extends ChannelInitializer<Channel> {

  private static final ChannelHandler INSTANCE = new StatelessHandler();

  @Override

  public void initChannel(Channel ch) {

    ch.pipeline().addLast(INSTANCE);

  }

}

22.Remove ChannelHandler once not needed anymore

public class OneTimeHandler extends ChannelInboundHandlerAdapter {

  @Override

  public void channelActive(ChannelHandlerContext ctx) {

    doOneTimeAction();

    ctx.channel().pipeline().remove(this); //Remove ChannelHandler once not needed anymore.

   }

}

This keeps the ChannelPipeline as short as possible and so eliminate overhead of traversing as much as possible.

23.Pass custom events through ChannelPipeline

Your custom events

public enum CustomEvents {

  MyCustomEvent

}

public class CustomEventHandler extends ChannelInboundHandlerAdapter {

  @Override

  public void userEventTriggered(ChannelHandlerContext ctx, Object evt) {

    if (evt == MyCustomEvent) { // do something}

  }

}

ChannelPipeline pipeline = channel.pipeline();

pipeline.fireUserEventTriggered(MyCustomEvent);

--Good fit for handshake notifications and more

24.Use proper buffer type in MessageToByteEncoder

public class EncodeActsOnByteArray extends MessageToByteEncoder<YourMessage> {

  public EncodeActsOnByteArray() { super(false); //Ensure heap buffers are used when pass into encode(...) method. This way you can access the backing array directly

  }

  @Override

  public encode(ChannelHandlerContext ctx, YourMessage msg, ByteBuf out) {

    byte[] array = out.array(); //Access the backing array and also calculate offset

    int offset = out.arrayOffset() + out.writerIndex();

    out.writeIndex(out.writerIndex() + encode(msg, array, offset)); //Update writerIndex to reflect written bytes

  }

  private int encode(YourMessage msg, byte[] array, int offset, int len) { ... }

}

--This saves extra byte copies.

25.To auto-read or not to auto-read

By default Netty will keep on reading data from the Channel once something is ready.

--Need more fine grained control ?

channel.config().setAutoRead(false); //Disable auto read == no more data will be read automatically from this Channel.

channel.read(); //Tell the Channel to do one read operation once new data is ready

channel.config().setAutoRead(true); //Enable again auto read == Netty will automatically read again

--This can also be quite useful when writing proxy like applications!

26.Native stuff in Netty 4

OpenSSL based SslEngine to reduce memory usage and latency.

Native transport for Linux using Epoll ET for more performance and less CPU usage.

Native transport also supports SO_REUSEPORT and TCP_CORK :)

27.Switching to native transport is easy

Using NIO transport

Bootstrap bootstrap = new Bootstrap().group(new NioEventLoopGroup());

bootstrap.channel(NioSocketChannel.class);

Using native transport

Bootstrap bootstrap = new Bootstrap().group(new EpollEventLoopGroup());

bootstrap.channel(EpollSocketChannel.class);