How to Build a Scalable Multiplexed Server With NIO

转载 2015年07月10日 22:52:09

How to Build a Scalable Multiplexed Server With NIO  


Architect a scalable, multiplexed Java Server using the New I/O (NIO) and Concurrency APIs

Building an NIO Server
Understanding the problem
Defining a solution
An NIO implementation

What Does a Server Do?
A server processes requests:
Receive a client request
Perform some request-specific task
Return a response
Multiple requests run concurrently
Client requests correspond to connections
Sequential requests may be sent on a single socket
Requests may contend for resources
Must tolerate slow, misbehaving or
unresponsive clients

Multiplexing Strategies
Poll each socket in turn
Impractical without non-blocking sockets
Inefficient, not fair and scales poorly
Only practical solution with blocking sockets
Stresses the thread scheduler, which limits scalability
Thread scheduler does readiness selection—inefficiently
Readiness selection
Efficient, but requires OS and Java VM support
Scales well, especially for many slow clients

Other Considerations
Multi-threading issues are magnified
Access controls may become a bottleneck
Non-obvious example: formatting text messages for logging
Potential for deadlocks
Per-thread overhead
Diminishing returns as threads/CPU ratio increases
Quality-of-service policy under load
Define acceptable performance criteria
Define what happens when threshold(s) are reached
Do nothing different, prioritize requests, queue new requests, reject new
requests, redirect to another server, and so on and so on...
Client profile
Ratio of connected clients to running requests
Can (or must) you tolerate malfunctioning or malicious clients?

The Reactor Pattern
Published in Pattern Languages of Program
Design, 1995, ISBN 0-201-6073-4
Paper by Prof. Douglas C. Schmidt *
Google for: Reactor Pattern
Describes the basic elements of a server
Gives us a vocabulary for this discussion

Reactor Pattern Participants
A reference to an event source, such as a socket
A state change that can occur on a Handle
Reacts to and interprets Events on Handles
Invokes Handlers for Events on Handles
Invoked to process an Event on a Handle
Dispatcher Flow (Single Threaded)
Dispatcher Flow (Multi-Threaded)

A Quick Diversion
Network connections are streams
If your code assumes structured reads, it’s broken
When reading:
You may only get some (or none) of the data
Structured messages will be fragmented
You must buffer bytes and reconstitute the structure
When writing:
The channel may not accept all you want to send
You must queue output data
Don’t spin in a handler waiting for output to drain
Perhaps it never will
Our server will be multi-threaded
Use the java.util.concurrent package (Java SE 5)
One select loop (Dispatcher)
Accepting new sockets is done elsewhere
We only care about input handlers
One input Handler per channel
No handler chains
Input and output processing are not directly coupled
Queuing is done by the framework
Input handlers do not enforce queuing policies

Let’s Quickly Review
Readiness Selection with NIO
Selector (Demultiplexer)
Holds a Set of keys representing ready channels
This is the “selected set” of keys
Events are added to but never removed from a key in this set
SelectionKey (Handle)
        Associates a Selector with a SelectableChannel
Holds set of events of interest for the channel
Events not in the interest set are ignored
Holds a set of triggered events as-of last select() call
Events persist until key is removed from the selected set
May hold an opaque Object reference for your use

Reactor Pattern Mapped to NIO
SelectionKey.OP_READ, etc
Dispatcher + iterate Selector.selectedKeys()
An instance of Runnable or Callable

The Selector class is kind of cranky about threads
While a thread is sleeping in select(), many
Selector and SelectionKey methods can block
indefinitely if invoked from a different thread
Use a guard object to handshake
Selection thread grabs then releases the guard
Other threads wishing to change Selector state
Lock the guard object
Wakeup the selector
Do whatever (eg: key.interestOps())
Release the guard lock

How to Make a AutoBuild System with Git Server

personal blog: emai:   今天给大家分享一下最近自己的成果, ...
  • tony7758
  • tony7758
  • 2011年11月29日 00:58
  • 2894

How to Build an Economic Model in Your Spare Time (如何利用业余时间建立经济模型)

如何利用业余时间建立经济模型 How to Build an Economic Model in Your Spare Time by Hal R. Varian UC Berkeley Decemb...
  • metababy
  • metababy
  • 2008年10月20日 21:25
  • 3211

Tutorial #Facebook Relay文档翻译#

Tutorial #Facebook Relay#文档翻译
  • tywinstark
  • tywinstark
  • 2015年09月23日 21:22
  • 1682

XGBoost: A Scalable Tree Boosting System

[ 论文阅读地址 ]1. 背景知识介绍函数的风险  给定关于XX 和YY 的空间,学习一个函数h:X→Yh:X\to Y ,函数的输入x∈Xx \in X ,输出y∈Yy \in Y。要学习函数hh,...
  • u012476429
  • u012476429
  • 2016年11月18日 16:43
  • 1328

How to Build Your Own Blockchain Part 2 — Syncing Chains From Different Nodes

Welcome to part 2 of the JackBlockChain, where I write some code to introduce the ability for differ...
  • omnispace
  • omnispace
  • 2018年01月29日 12:55
  • 95

how to build a git server

see link:
  • GW569453350game
  • GW569453350game
  • 2016年08月11日 16:50
  • 2716

论文阅读-XGBoost: A Scalable Tree Boosting System

Tianqi Chen大神提出了一种可扩展性的端到端基于树的boosting系统,这个系统可以处理稀疏性数据,通过分布式加权直方图算法去近似学习树,这个系统也提供基于缓存的加速模式、数据压缩、分片功能...
  • u014411730
  • u014411730
  • 2017年12月15日 20:50
  • 319

How to Build a Python Bot That Can Play Web Games

  • lifaming15
  • lifaming15
  • 2015年03月09日 11:25
  • 1417

netty websocket chatroom

we can use netty to build a chatroom using its websocket.          First, we can use SSL to make it...
  • hnjdx814733170
  • hnjdx814733170
  • 2016年06月07日 14:15
  • 273

ceph翻译 Ceph: A Scalable, High-Performance Distributed File System

Ceph: A Scalable, High-Performance Distributed File System Ceph:一个可扩展,高性能分布式文件系统 Sage A. Weil Sco...
  • XingKong_678
  • XingKong_678
  • 2016年05月28日 20:15
  • 891
您举报文章:How to Build a Scalable Multiplexed Server With NIO