How to Build a Scalable Multiplexed Server With NIO

转载 2015年07月10日 22:52:09

How to Build a Scalable Multiplexed Server With NIO  


Architect a scalable, multiplexed Java Server using the New I/O (NIO) and Concurrency APIs

Building an NIO Server
Understanding the problem
Defining a solution
An NIO implementation

What Does a Server Do?
A server processes requests:
Receive a client request
Perform some request-specific task
Return a response
Multiple requests run concurrently
Client requests correspond to connections
Sequential requests may be sent on a single socket
Requests may contend for resources
Must tolerate slow, misbehaving or
unresponsive clients

Multiplexing Strategies
Poll each socket in turn
Impractical without non-blocking sockets
Inefficient, not fair and scales poorly
Only practical solution with blocking sockets
Stresses the thread scheduler, which limits scalability
Thread scheduler does readiness selection—inefficiently
Readiness selection
Efficient, but requires OS and Java VM support
Scales well, especially for many slow clients

Other Considerations
Multi-threading issues are magnified
Access controls may become a bottleneck
Non-obvious example: formatting text messages for logging
Potential for deadlocks
Per-thread overhead
Diminishing returns as threads/CPU ratio increases
Quality-of-service policy under load
Define acceptable performance criteria
Define what happens when threshold(s) are reached
Do nothing different, prioritize requests, queue new requests, reject new
requests, redirect to another server, and so on and so on...
Client profile
Ratio of connected clients to running requests
Can (or must) you tolerate malfunctioning or malicious clients?

The Reactor Pattern
Published in Pattern Languages of Program
Design, 1995, ISBN 0-201-6073-4
Paper by Prof. Douglas C. Schmidt *
Google for: Reactor Pattern
Describes the basic elements of a server
Gives us a vocabulary for this discussion

Reactor Pattern Participants
A reference to an event source, such as a socket
A state change that can occur on a Handle
Reacts to and interprets Events on Handles
Invokes Handlers for Events on Handles
Invoked to process an Event on a Handle
Dispatcher Flow (Single Threaded)
Dispatcher Flow (Multi-Threaded)

A Quick Diversion
Network connections are streams
If your code assumes structured reads, it’s broken
When reading:
You may only get some (or none) of the data
Structured messages will be fragmented
You must buffer bytes and reconstitute the structure
When writing:
The channel may not accept all you want to send
You must queue output data
Don’t spin in a handler waiting for output to drain
Perhaps it never will
Our server will be multi-threaded
Use the java.util.concurrent package (Java SE 5)
One select loop (Dispatcher)
Accepting new sockets is done elsewhere
We only care about input handlers
One input Handler per channel
No handler chains
Input and output processing are not directly coupled
Queuing is done by the framework
Input handlers do not enforce queuing policies

Let’s Quickly Review
Readiness Selection with NIO
Selector (Demultiplexer)
Holds a Set of keys representing ready channels
This is the “selected set” of keys
Events are added to but never removed from a key in this set
SelectionKey (Handle)
        Associates a Selector with a SelectableChannel
Holds set of events of interest for the channel
Events not in the interest set are ignored
Holds a set of triggered events as-of last select() call
Events persist until key is removed from the selected set
May hold an opaque Object reference for your use

Reactor Pattern Mapped to NIO
SelectionKey.OP_READ, etc
Dispatcher + iterate Selector.selectedKeys()
An instance of Runnable or Callable

The Selector class is kind of cranky about threads
While a thread is sleeping in select(), many
Selector and SelectionKey methods can block
indefinitely if invoked from a different thread
Use a guard object to handshake
Selection thread grabs then releases the guard
Other threads wishing to change Selector state
Lock the guard object
Wakeup the selector
Do whatever (eg: key.interestOps())
Release the guard lock


在工作中自开发了一套java系统,这个系统的功能是通过quartz定时生成报表,用户通过登录系统来查看报表。IBM公司也有这样一个系统。 在开发这个系统的时候为了满足快速迭代的要求,需要实现自动打包...
  • sinat_33186733
  • sinat_33186733
  • 2016年03月16日 23:02
  • 6746

react-native 环境搭建遇到问题

1.   D:\Android\WorkSpace\AwesomeProject>react-native run-android JS server already running. Buildin...
  • shuai123456
  • shuai123456
  • 2016年07月14日 16:36
  • 9315


1、导言 折腾mongodb几个小时终于有结果了。呃!现在就简单总结一下。 其实我的需求很简单,就是在C++代码中调用mongodb的库函数,也就是要得到mongoclient.lib。本来想直接下载...
  • BaiWfg2
  • BaiWfg2
  • 2014年07月22日 17:14
  • 7945

How to Build a Scalable ETL Pipeline with Kafka Connect(转)

Apache Kafka is a high-throughput distributed message system that is being adopted by hundreds of co...
  • GK_kk
  • GK_kk
  • 2017年03月15日 16:47
  • 159

How to build a product with InstallShield

How to build a product with InstallShield: put all build script in same folder. Example: "b...
  • zhen_zhang20
  • zhen_zhang20
  • 2014年05月26日 09:57
  • 610

How to build CP2K with Intel Compiler 12.1, OpenMPI-1.4.3 and Infiniband Support
  • tengh
  • tengh
  • 2012年06月07日 17:11
  • 1117

How to Build ffmpeg with NDK r9

  • Mephisto0568
  • Mephisto0568
  • 2014年09月15日 19:43
  • 403

DIY BROADCAST : How to build your own Internet TV Channel with Open-Source & other goodies

DIY BROADCAST : How to build your own Internet TV Channel with Open-Source & other goodies
  • charleslei
  • charleslei
  • 2016年12月28日 10:44
  • 893

Translation:How to build Tesseract 3.03 with Visual Studio 2013 (翻译《VS2013 建立tesseract3.03工程》)

How to build tesseract 3.03 with Visual Studio 2013                       Visual studio2013 建立tesser...
  • u013985291
  • u013985291
  • 2016年03月24日 21:33
  • 789

How to build applications with OpenCV inside the Microsoft Visual Studio

如何在Microsoft Visual内构建OpenCV应用程序在Visual Studio中的一个项目的基础项目也称为一个解决方案。一个解决方案也可以包含多个项目。这一点是我以前没有遇到过的。项目是...
  • sinat_25926805
  • sinat_25926805
  • 2015年02月09日 17:24
  • 674
您举报文章:How to Build a Scalable Multiplexed Server With NIO