How to Build a Scalable Multiplexed Server With NIO

How to Build a Scalable Multiplexed Server With NIO  

转自 JAVA NIO作者PPT

Architect a scalable, multiplexed Java Server using the New I/O (NIO) and Concurrency APIs


Building an NIO Server
Understanding the problem
Defining a solution
An NIO implementation

What Does a Server Do?
A server processes requests:
Receive a client request
Perform some request-specific task
Return a response
Multiple requests run concurrently
Client requests correspond to connections
Sequential requests may be sent on a single socket
Requests may contend for resources
Must tolerate slow, misbehaving or
unresponsive clients

Multiplexing Strategies
Poll each socket in turn
Impractical without non-blocking sockets
Inefficient, not fair and scales poorly
Thread-per-socket
Only practical solution with blocking sockets
Stresses the thread scheduler, which limits scalability
Thread scheduler does readiness selection—inefficiently
Readiness selection
Efficient, but requires OS and Java VM support
Scales well, especially for many slow clients

Other Considerations
Multi-threading issues are magnified
Access controls may become a bottleneck
Non-obvious example: formatting text messages for logging
Potential for deadlocks
Per-thread overhead
Diminishing returns as threads/CPU ratio increases
Quality-of-service policy under load
Define acceptable performance criteria
Define what happens when threshold(s) are reached
Do nothing different, prioritize requests, queue new requests, reject new
requests, redirect to another server, and so on and so on...
Client profile
Ratio of connected clients to running requests
Can (or must) you tolerate malfunctioning or malicious clients?

The Reactor Pattern
Published in Pattern Languages of Program
Design, 1995, ISBN 0-201-6073-4
Paper by Prof. Douglas C. Schmidt *
Google for: Reactor Pattern
Describes the basic elements of a server
Gives us a vocabulary for this discussion

Reactor Pattern Participants
Handle
A reference to an event source, such as a socket
Event
A state change that can occur on a Handle
Demultiplexer
Reacts to and interprets Events on Handles
Dispatcher
Invokes Handlers for Events on Handles
Handler
Invoked to process an Event on a Handle
句柄：
   时间源的引用，例如socket
事件：
   在socket上面发生的状态改变
多路复用器：
   处理在句柄上面的事件
分发器：
   对于在socket上面的事件调用相应处理器
处理器：
   调用函数处理在socket上面的事件
   
Dispatcher Flow (Single Threaded)
Dispatcher Flow (Multi-Threaded)


A Quick Diversion
Network connections are streams
If your code assumes structured reads, it’s broken
When reading:
You may only get some (or none) of the data
Structured messages will be fragmented
You must buffer bytes and reconstitute the structure
When writing:
The channel may not accept all you want to send
You must queue output data
Don’t spin in a handler waiting for output to drain
Perhaps it never will
Our server will be multi-threaded
Use the java.util.concurrent package (Java SE 5)
One select loop (Dispatcher)
Accepting new sockets is done elsewhere
We only care about input handlers
One input Handler per channel
No handler chains
Input and output processing are not directly coupled
Queuing is done by the framework
Input handlers do not enforce queuing policies

Let’s Quickly Review
Readiness Selection with NIO
Selector (Demultiplexer)
Holds a Set of keys representing ready channels
This is the “selected set” of keys
Events are added to but never removed from a key in this set
SelectionKey (Handle)
        Associates a Selector with a SelectableChannel
Holds set of events of interest for the channel
Events not in the interest set are ignored
Holds a set of triggered events as-of last select() call
Events persist until key is removed from the selected set
May hold an opaque Object reference for your use

Reactor Pattern Mapped to NIO
Handle
SelectionKey
Event
SelectionKey.OP_READ, etc
Demultiplexer
Selector
Dispatcher
Selector.select() + iterate Selector.selectedKeys()
Handler
An instance of Runnable or Callable

The Selector class is kind of cranky about threads
While a thread is sleeping in select(), many
Selector and SelectionKey methods can block
indefinitely if invoked from a different thread
Use a guard object to handshake
Selection thread grabs then releases the guard
Other threads wishing to change Selector state
Lock the guard object
Wakeup the selector
Do whatever (eg: key.interestOps())
Release the guard lock