在大型网络应用程序的场景中,服务端程序得同时面对成千上万的客户请求,如果给每一个客户请求分配单独的进程或者线程,将会对服务端造成极大的内存和CPU压力。幸运的是,python提供了select模块以解决这个问题。
1. select模块简介
通过select模块python可以接入操作系统的poll()和select()功能,从而实现基于多个文件描述符的异步I/O功能。
该模块定义了以下基本类:
select.error()
: select模块的异常类;
select.poll()
: 返回一个支持注册和注销文件描述符的轮询对象,然后对他们的IO事件进行轮询;
select.kqueue()
:返回一个内核队列(kernel queue)对象;
select.kevent(ident, filter=KQ_FILTER_READ, flags=KQ_EV_ADD, fflags=0, data=0, udata=0)
: 返回内核事件(kernel event)对象;
select.select(rlist, wlist, xlist[, timeout])
:这是Unix的select()系统调用的简单接口,前三个参数是可等待的对象,可以是代表文件描述符的整数或者包含fileno()方法的对象。具体的:
rlist
: 等待直到可读wlist
:等待直到可写xlist
:等待直到发生异常
可选参数timeout
定义超时时间。
2. 代码示例
下面的例子使用select模块的select方法,使的服务器和客户端之间收发信息而不阻塞所有通话时间:
import select
import socket
import sys
import signal
import cPickle
import struct
import argparse
SERVER_HOST = 'localhost'
CHAT_SERVER_NAME = 'server'
# Some utilities
def send(channel, *args):
buffer = cPickle.dumps(args)
value = socket.htonl(len(buffer))
size = struct.pack("L",value)
channel.send(size)
channel.send(buffer)
def receive(channel):
size = struct.calcsize("L")
size = channel.recv(size)
try:
size = socket.ntohl(struct.unpack("L", size)[0])
except struct.error, e:
return ''
buf = ""
while len(buf) < size:
buf = channel.recv(size - len(buf))
return cPickle.loads(buf)[0]
class ChatServer(object):
""" An example chat server using select """
def __init__(self, port, backlog=5):
self.clients = 0
self.clientmap = {}
self.outputs = [] # list output sockets
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Enable re-using socket address
self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.server.bind((SERVER_HOST, port))
print 'Server listening to port: %s ...' %port
self.server.listen(backlog)
# Catch keyboard interrupts
signal.signal(signal.SIGINT, self.sighandler)
def sighandler(self, signum, frame):
""" Clean up client outputs"""
# Close the server
print 'Shutting down server...'
# Close existing client sockets
for output in self.outputs:
output.close()
self.server.close()
def get_client_name(self, client):
""" Return the name of the client """
info = self.clientmap[client]
host, name = info[0][0], info[1]
return '@'.join((name, host))
def run(self):
inputs = [self.server, sys.stdin]
self.outputs = []
running = True
while running:
try:
readable, writeable, exceptional = \
select.select(inputs, self.outputs, [])
except select.error, e:
break
for sock in readable:
if sock == self.server:
# handle the server socket
client, address = self.server.accept()
print "Chat server: got connection %d from %s" % (client.fileno(), address)
# Read the login name
cname = receive(client).split('NAME: ')[1]
# Compute client name and send back
self.clients += 1
send(client, 'CLIENT: ' + str(address[0]))
inputs.append(client)
self.clientmap[client] = (address, cname)
# Send joining information to other clients
msg = "\n(Connected: New client (%d) from %s)" %\
(self.clients, self.get_client_name(client))
for output in self.outputs:
send(output, msg)
self.outputs.append(client)
elif sock == sys.stdin:
# handle standard input
junk = sys.stdin.readline()
running = False
else:
# handle all other sockets
try:
data = receive(sock)
if data:
# Send as new client's message...
msg = '\n#[' +self.get_client_name(sock) + ']>>' + data
# Send data to all except ourself
for output in self.outputs:
if output != sock:
send(output, msg)
else:
print "Chat server: %d hung up" % sock.fileno()
self.clients -= 1
sock.close()
inputs.remove(sock)
self.outputs.remove(sock)
# Sending client leaving info to others
msg = "\n(Now hung up: Client from %s)" % self.get_client_name(sock)
for output in self.outputs:
send(output, msg)
except socket.error, e:
# Remove
inputs.remove(sock)
self.outputs.remove(sock)
self.server.close()
服务器程以系统输入和服务器套接字作为select方法的`rlist`参数,服务器程序不断查询可读列表`readable`,当self.server变为可读时,说明有客户端请求到达服务端,这时调用server的accept参数接受客户端连接并进行相应处理。客户端类似。
(例子摘自Python Network Programming Cookbook)