python socketserver 10万并发_python学习之socketserver实现并发

前面学习了socket的相关知识，但是只能处理单线程的，当然我们也可以使用python多线程模块实现多线程并发，python中socketserver模块实现了并发相关操作，本文主要记录一下学习过程。

服务端代码如下：#1、自定义一个类#2、在类中重写handle方法(该方法中实现自己业务逻辑)

importsocketserverclassMyserver(socketserver.BaseRequestHandler):defhandle(self):print('conn is:', self.request) #conn

print('addr is:', self.client_address) #addr

whileTrue:try:

data=self.request.recv(1024)if not data:break

print('收到客户端的消息是', data, self.client_address)#发消息

self.request.sendall(data.upper())exceptException as e:print(e)break

if __name__ == '__main__':

s= socketserver.ThreadingTCPServer(('127.0.0.1', 9090), Myserver)#实例化,会执行__init__方法(按照查找顺序去查找)，

s.serve_forever()

客户端代码可以直接用前面的

importsocket

sk=socket.socket()

addr= ('127.0.0.1',9090)

sk.connect(addr)whileTrue:

inp= input('>>>')if inp == 'q':breaksk.send(bytes(inp,'utf8'))

data= sk.recv(1024)print(str(data,'utf8'))

sk.close()

客户端

服务端socketserver 内部主要流程和相关源码分析如下：

1、s = socketserver.ThreadingTCPServer(('127.0.0.1', 9090), Myserver)进行了实例化

ThreadingTCPServer(('127.0.0.1', 9090), Myserver)实例化，会执行__init__方法(按照查找顺序去查找)，执行TCPServer类中__init__方法(TCPServer类中调用了BaseServer类中__init__方法(BaseServer.__init__))

def __init__(self, server_address, RequestHandlerClass):"""Constructor. May be extended, do not override."""self.server_address=server_address

self.RequestHandlerClass= RequestHandlerClass#就是我们自己定义的Myserver类

self.__is_shut_down =threading.Event()

self.__shutdown_request = False

BaseServer类中__init__代码

def __init__(self, server_address, RequestHandlerClass, bind_and_activate=True):"""Constructor. May be extended, do not override."""BaseServer.__init__(self, server_address, RequestHandlerClass)

self.socket=socket.socket(self.address_family,

self.socket_type)#此处看到熟悉的代码，即创建了socket对象

ifbind_and_activate:try:

self.server_bind()

self.server_activate()except:

self.server_close()raise

TCPServer类中__init__代码

分析TCPServer类中__init__代码可以看到做了如下操作1、调用了BaseServer类中__init__方法(BaseServer类中__init__中主要就是进行了一些属性封装操作)2、创建socket对象3、执行server_bind和server_activate方法(也是按照类继承的查找顺序去查找执行哪个类中的该方法，此时执行的就是TCPServer类中这两个方法)4、server_bind中主要就是self.socket.bind(self.server_address)即bind ip和port5、server_activate中就是self.socket.listen(self.request_queue_size)即listen

由上述分析可知s = socketserver.ThreadingTCPServer(('127.0.0.1', 9090), Myserver)执行后主要就是创建socket对象、bind、listen，并且把我们自定义的类赋值给self.RequestHandlerClass

2、执行s.serve_forever()

s.serve_forever()即实例调用类方法(分析可知执行的是BaseServer类中serve_forever方法)，可以看到serve_forever中调用self._handle_request_noblock()方法，self._handle_request_noblock()

中调用self.process_request(request, client_address)方法(按照查找顺序查找执行的是哪个类中self.process_request)，分析可知，执行的是ThreadingMixIn类中的process_request方法，self.process_request中

执行self.process_request_thread方法，self.process_request_thread执行self.finish_request(request, client_address)(按照查找顺序去执行，可以看到执行的是BaseServer中finish_request)，self.finish_request执行

self.RequestHandlerClass(request, client_address, self)，可以看到就是执行我们最初自定义的那个类，此时就是对Myserver类进行了实例化，也会先执行__init__方法,通过分析可知执行的是Myserver

继承的类BaseRequestHandler中__init__方法，分析可以看到执行了handle()(就是我们自己定义的Myserver类中的handle方法，进入到我们自定义的逻辑)

注意:通过self去调用各种类中的方法时，类继承时查找方法的顺序，在该分析中self永远代指s这个实例

def serve_forever(self, poll_interval=0.5):"""Handle one request at a time until shutdown.

Polls for shutdown every poll_interval seconds. Ignores

self.timeout. If you need to do periodic tasks, do them in

another thread."""self.__is_shut_down.clear()try:#XXX: Consider using another file descriptor or connecting to the

#socket to wake this up instead of polling. Polling reduces our

#responsiveness to a shutdown request and wastes cpu at all other

#times.

with _ServerSelector() as selector:

selector.register(self, selectors.EVENT_READ)while not self.__shutdown_request:

ready=selector.select(poll_interval)ifready:

self._handle_request_noblock()

self.service_actions()finally:

self.__shutdown_request =False

self.__is_shut_down.set()

BaseServer类中serve_forever方法

def_handle_request_noblock(self):"""Handle one request, without blocking.

I assume that selector.select() has returned that the socket is

readable before this function was called, so there should be no risk of

blocking in get_request()."""

try:

request, client_address=self.get_request()exceptOSError:return

ifself.verify_request(request, client_address):try:

self.process_request(request, client_address)exceptException:

self.handle_error(request, client_address)

self.shutdown_request(request)except:

self.shutdown_request(request)raise

else:

self.shutdown_request(request)

BaseServer类中handle_request_noblock方法

classThreadingMixIn:"""Mix-in class to handle each request in a new thread."""

#Decides how threads will act upon termination of the

#main process

daemon_threads =Falsedefprocess_request_thread(self, request, client_address):"""Same as in BaseServer but as a thread.

In addition, exception handling is done here."""

try:

self.finish_request(request, client_address)exceptException:

self.handle_error(request, client_address)finally:

self.shutdown_request(request)defprocess_request(self, request, client_address):"""Start a new thread to process the request."""t= threading.Thread(target =self.process_request_thread,

args=(request, client_address))

t.daemon=self.daemon_threads

t.start()

ThreadingMixIn类中方法

deffinish_request(self, request, client_address):"""Finish one request by instantiating RequestHandlerClass."""self.RequestHandlerClass(request, client_address, self)

BaseServer中finish_request

#可以看到执行了handle()，就是我们自己定义的Myserver类中的handle方法

def __init__(self, request, client_address, server):

self.request=request

self.client_address=client_address

self.server=server

self.setup()try:

self.handle()finally:

self.finish()

BaseRequestHandler类__init__

整体的主要流程如下

1、执行 TCPServer.init 方法，创建服务端Socket对象并绑定 IP 和 端口2、执行 BaseServer.init 方法，将我们自定义的类赋值给self.RequestHandlerClass3、执行 BaseServer.server_forever 方法，While 循环一直监听是否有客户端请求到达

有客户端请求到来时1、执行 ThreadingMixIn.process_request 方法，创建一个 “线程” 用来处理请求2、执行 ThreadingMixIn.process_request_thread 方法3、执行 BaseServer.finish_request 方法(执行 self.RequestHandlerClass() ，即实例化会执行__init__,构造方法中执行了self.handle()此时执行的就是我们自定义类中的handle方法)