How to Build a Scalable Multiplexed Server With NIO

转载 2015年07月10日 22:52:09

How to Build a Scalable Multiplexed Server With NIO  


Architect a scalable, multiplexed Java Server using the New I/O (NIO) and Concurrency APIs

Building an NIO Server
Understanding the problem
Defining a solution
An NIO implementation

What Does a Server Do?
A server processes requests:
Receive a client request
Perform some request-specific task
Return a response
Multiple requests run concurrently
Client requests correspond to connections
Sequential requests may be sent on a single socket
Requests may contend for resources
Must tolerate slow, misbehaving or
unresponsive clients

Multiplexing Strategies
Poll each socket in turn
Impractical without non-blocking sockets
Inefficient, not fair and scales poorly
Only practical solution with blocking sockets
Stresses the thread scheduler, which limits scalability
Thread scheduler does readiness selection—inefficiently
Readiness selection
Efficient, but requires OS and Java VM support
Scales well, especially for many slow clients

Other Considerations
Multi-threading issues are magnified
Access controls may become a bottleneck
Non-obvious example: formatting text messages for logging
Potential for deadlocks
Per-thread overhead
Diminishing returns as threads/CPU ratio increases
Quality-of-service policy under load
Define acceptable performance criteria
Define what happens when threshold(s) are reached
Do nothing different, prioritize requests, queue new requests, reject new
requests, redirect to another server, and so on and so on...
Client profile
Ratio of connected clients to running requests
Can (or must) you tolerate malfunctioning or malicious clients?

The Reactor Pattern
Published in Pattern Languages of Program
Design, 1995, ISBN 0-201-6073-4
Paper by Prof. Douglas C. Schmidt *
Google for: Reactor Pattern
Describes the basic elements of a server
Gives us a vocabulary for this discussion

Reactor Pattern Participants
A reference to an event source, such as a socket
A state change that can occur on a Handle
Reacts to and interprets Events on Handles
Invokes Handlers for Events on Handles
Invoked to process an Event on a Handle
Dispatcher Flow (Single Threaded)
Dispatcher Flow (Multi-Threaded)

A Quick Diversion
Network connections are streams
If your code assumes structured reads, it’s broken
When reading:
You may only get some (or none) of the data
Structured messages will be fragmented
You must buffer bytes and reconstitute the structure
When writing:
The channel may not accept all you want to send
You must queue output data
Don’t spin in a handler waiting for output to drain
Perhaps it never will
Our server will be multi-threaded
Use the java.util.concurrent package (Java SE 5)
One select loop (Dispatcher)
Accepting new sockets is done elsewhere
We only care about input handlers
One input Handler per channel
No handler chains
Input and output processing are not directly coupled
Queuing is done by the framework
Input handlers do not enforce queuing policies

Let’s Quickly Review
Readiness Selection with NIO
Selector (Demultiplexer)
Holds a Set of keys representing ready channels
This is the “selected set” of keys
Events are added to but never removed from a key in this set
SelectionKey (Handle)
        Associates a Selector with a SelectableChannel
Holds set of events of interest for the channel
Events not in the interest set are ignored
Holds a set of triggered events as-of last select() call
Events persist until key is removed from the selected set
May hold an opaque Object reference for your use

Reactor Pattern Mapped to NIO
SelectionKey.OP_READ, etc
Dispatcher + iterate Selector.selectedKeys()
An instance of Runnable or Callable

The Selector class is kind of cranky about threads
While a thread is sleeping in select(), many
Selector and SelectionKey methods can block
indefinitely if invoked from a different thread
Use a guard object to handshake
Selection thread grabs then releases the guard
Other threads wishing to change Selector state
Lock the guard object
Wakeup the selector
Do whatever (eg: key.interestOps())
Release the guard lock


How to Build a Scalable ETL Pipeline with Kafka Connect(转)

Apache Kafka is a high-throughput distributed message system that is being adopted by hundreds of co...
  • GK_kk
  • GK_kk
  • 2017年03月15日 16:47
  • 122

How to build a product with InstallShield

How to build a product with InstallShield: put all build script in same folder. Example: "b...

How to build CP2K with Intel Compiler 12.1, OpenMPI-1.4.3 and Infiniband Support
  • tengh
  • tengh
  • 2012年06月07日 17:11
  • 1074

How to Build ffmpeg with NDK r9


DIY BROADCAST : How to build your own Internet TV Channel with Open-Source & other goodies

DIY BROADCAST : How to build your own Internet TV Channel with Open-Source & other goodies

Translation:How to build Tesseract 3.03 with Visual Studio 2013 (翻译《VS2013 建立tesseract3.03工程》)

How to build tesseract 3.03 with Visual Studio 2013                       Visual studio2013 建立tesser...

How to build applications with OpenCV inside the Microsoft Visual Studio

如何在Microsoft Visual内构建OpenCV应用程序在Visual Studio中的一个项目的基础项目也称为一个解决方案。一个解决方案也可以包含多个项目。这一点是我以前没有遇到过的。项目是...

How to set up tomcat with server authentication and client certificate authentication enabled

How to set up tomcat with server authentication and client certificate authentication enabled? When...
您举报文章:How to Build a Scalable Multiplexed Server With NIO