运维小白的Python之路——socket编程

一、基础概述

socket通常也称作“套接字”，用户描述IP地址和端口，是一个通信链的句柄，应用程序通常通过“套接字”，想网络发出请求或者应答网络请求。
socket起源于Unix，而Unix\Linux的基本哲学之一就是“一切皆文件”，对于文件用“打开”，“读写”，“关闭”模式来操作。socket就是该模式的一个实现，socket即是一种特殊的文件，一些socket函数就是对其进行的一种操作（读写、打开和关闭）

socket和文件的区别：
1.file是针对某个文件进行的操作——打开、关闭、读写
2.socket是针对服务端和客户端socket进行打开读写和关闭

socket小练习
server端

#!/usr/bin/env python3
import socket

ip_port = ("127.0.0.1", 8888)

sk = socket.socket() #创建socket实例
sk.bind(ip_port) #绑定IP，端口
sk.listen(5) #监听端口，最大连接数5
print("server is waitting...")
conn,addr = sk.accept() #等待客户端的链接如果没有收到消息就在这里阻塞
while True:
    client_data = str(conn.recv(1024),encoding = "utf8") #接受客户端的信息
    print(client_data)
    conn.sendall(bytes("这是server端的回话",encoding = "utf8"))
conn.close() #断开链接

client端

#!/usr/bin/env python3

import socket

ip_port = ("127.0.0.1", 8888)

sk = socket.socket() #创建socket实例
sk.connect(ip_port) #链接服务端
while True:
    user_input = input("请输入")
    sk.sendall(bytes(user_input,encoding = "utf8")) #向服务端发送信息
    server_reply = sk.recv(1024) #接受服务端的信息
    print(str(server_reply,encoding = "utf8"))
sk.close()

二、内部方法

server = socket.socket(socket.AF_INET, socket.SOCK_StrEAM, 0)
参数一：地址簇
socket.AF_INET IPv4(默认)
socket.AF_INEt6
socket.UNIX 只能用于单一的Unix系统进程间通信
参数二：类型
socket.SOCK_STREAM 流式socket，用户TCP协议
socket.SOCK_DGRAM 数据报式socket， 用于UDP协议
socket.SOCK——RAW 原始套接字。普通套接字无法术力ICMP、IGMP等网络报文，而SOCK_RAW可以；其次，SOCK_RAW也可以处理特殊的IPv4报文;此外，利用原始套接字，可以通过IP_HDRINCL套接字选项由用户构造IP头。
socket.SOCK_RDM 是一种可靠的UPD形式，即保证数据的传输但是不保证数据的顺序。SOCK_RAM用来供对原始数据的低级访问，在需要执行某些特殊操作时使用，如大宋ICMP报文。SOCK_RAM通常尽限于高级用户或管理员运行程序的程序使用。
socket.SOCK_SEPQACKET 可靠的连续数据包服务
参数三：协议
0 （默认）与特定的地址家族相关的协议，如果是0，则系统会根据地址格式和桃姐类别，自动选择一个合适的协议。
UPD演示

server端
#!/usrbin/env pyrhon3
import socket

ip_port = ('127.0.0.1', 8888)

server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)
server.bind(ip_port)

while True:
    data = str(server.recv(1024), encoding = "utf8")
    print(data)
server.close()

client端
#!/usr/bin/env python3

import socket
ip_port = ('127.0.0.1', 8888)

client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)
while True:
    inp = bytes(input("请输入").strip(), encoding = "utf8")
    if inp == "exit":
        break
    client.sendto(inp, ip_port)
client.close()

sk.bind(address)

sk.bind(address) 将套接字绑定到地址。address的地址格式取决于地址簇，在AF_INET下，以元组(host,ip)的形式表示地址。

sk.listen(backlog)

开始监听传入链接。backlog制定在拒绝链接之前，可以挂起的最大连接数量。
backlog = 5，表示内核已经接到了链接请求，但是服务器还没有调用accept进行处理的链接个数最大为5，这个之不能无限大，因为要在内核中维护链接队列

sk.setblocking(bool)

是否阻塞（默认是True），如果设置成False，那么accept和recv在没有接收到数据的时候，不阻塞，直接报错。

sk.accept()

接受链接并返回(conn,address)，其中conn是新的套接字对象，可以用来接收和发送数据。address是链接客户端的地址。
接收TCP客户的链接（阻塞式）等待链接的到来

sk.connect(address)

连接到address处的套接字。一般，address的格式为元组(hostname,port)，如果链接出错，返回socket.error错误。

sk.connect_ex(address)

同上，只不过会有返回值，链接成功是返回0，链接失败时候返回编码。

sk.close()

关闭套接字

sk.recv(bufsize[,flag])

接收套接字的数据。数据一字符串的形式返回，bufsize指定最多可以接收的数量，flag提供有关消息的其他信息，通常可以忽略。

sk.recvfrom(bufsize[.flag])

与recv()类似，但是返回值是(data,address)。其中data是包含接收数据的字符串，address是发送数据的套接字地址。

sk.send(string[,flag])

将string中的数据发送到链接的套接字。返回值是要发送的字节数量，改数量可能小于srting的字节大小，即：可能未将指定的内容全部发送。

sk.sendall(string[,flag])

将string中的数据发送到链接的套接字，但在返回之前会尝试发送所有的数据。成功返回None，失败则抛出异常。
内部通过递归调用send，将所有的数据发送出去。

sk.sendto(string[,flag],address)

将数据发送到套接字，address是一个(ipaddr, port)元组，是定原远程地址。返回值是大宋的字节数。该函数主要用于UDP协议。

sk.settimeout(timeout)

设置套接字操作超时，timeout十一二浮点数，单位是秒。值为None表示没有超时日期。一般超时日期应该在刚创建套接字时折尺，因为他们可能用于链接的操作（如client操作最多等待5S）

sk.getpeername()

返回套接字的远程地址。返回值通常是元组(ipaddr, port)

sk.getsockname()

返回自己的地址，返回值通常是个元组(ipaddr, port)

sk.fileno()

套接字的文件描述符

一个关于UDP的例子

server端
#!/usr/bin/emv python3
import socket
ip_port = ('127.0.0.1', 28888)
server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)
server.bind(ip_port)

while True:
    data,(host, port) = server.recvfrom(1024)
    print(data, host, port)
    server.sendto(bytes('ok', encoding = 'utf8'), (host, port))


client端
#!/usr/bin/env python3
import socket
ip_port = ('127.0.0.1', 28888)

client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)
while True:
    inp = input("数据:").strip()
    if inp == 'exit':
        break
    client.sendto(bytes(inp, encoding = "utf8"), ip_port)
    data = client.recvfrom(1024)
    print(data)

client.close()

实例（学说话）

实现了可重复的链接服务端

server端
#!/usr/bin/env python3

import socket

ip_port = ('127.0.0.1', 28888)
server = socket.socket()
server.bind(ip_port)
server.listen()

while True:
    link,ip = server.accept()
    while True:
        link.send("这个server会学说话".encode()) #对发送的数据进行编码，默认为utf8
        data = link.recv(1024).decode()
        print('data:',data)
        if not data:
            continue
        elif data == 'q':
            break
        print("从客户端接收的消息:",data)
        link.send(data.encode())
    link.close()

client端
 #!/usr/bin/env pyton3

import socket

ip_port = ('127.0.0.1', 28888)
client = socket.socket()
client.connect(ip_port)

data = client.recv(1024).decode() #对发来的数据进行解码，默认为utf8
print(data)
while True:
    user_input = input("请输入(q退出):")
    if not user_input:
        print("输入的不能为空")
        continue
    client.send(user_input.encode()) #当输入q时如果不先将q发到服务端再关闭socket，服务端会报(BrokenPipeError: [Errno 32] Broken pipe——管道破裂)
    if user_input == 'q':
        break
    server_res = client.recv(1024).decode()
    print(server_res)
client.close()

实例（简单的ssh）

client端
#!/usr/bin/env python3
import socket

ip_port = ('127.0.0.1', 28888)

client = socket.socket()
client.connect(ip_port)

title = client.recv(1027).decode()
print(title)

while True:
    user_input = input('请输入命令(q:退出)：')
    if not user_input:
        print("命令为空，请重新输入")
        continue
    client.send(user_input.encode())
    if user_input == "q":
        break
    res_len = int(client.recv(1024).decode())
    client.send("数据长度接受成功".encode())
    res = ''
    while len(res) < res_len:
        res += client.recv(1024).decode()
    print(res)

client.close()

server端
#!/usr/bin/env python3
import socket,os

ip_port = ('127.0.0.1', 28888)
server = socket.socket()
server.bind(ip_port)
server.listen()

while True:
    link,ip = server.accept()
    link.send("这个server是一个简单的ssh".encode())
    while True:
        client_cmd = link.recv(1024).decode()
        if client_cmd == "q":
            break
        res = os.popen(client_cmd).read()
        res_len = len(res)
        print(res_len)
        link.send(str(res_len).encode())
        res_succ = link.recv(1027).decode()
        link.send(res.encode())
    link.close()

socketserver模块

socketserver模块可以处理多进程的socket

ThreadingTCPServer

ThreadingTCPServer实现的Soket服务器内部会为每个client创建一个 “线程”，该线程用来和客户端进行交互。

1、ThreadingTCPServer基础

使用ThreadingTCPServer:
创建一个继承自 SocketServer.BaseRequestHandler 的类
类中必须定义一个名称为 handle 的方法
启动ThreadingTCPServer
socketserver实现服务端

只接收一个进程的情况
#!/usr/bin/env python3

import socketserver

class MyTCPHandler(socketserver.BaseRequestHandler):

    def handle(self):
        while True:#可以重复接收
            #等待客户端的链接，接收数据
            self.data = self.request.recv(1024).strip()
            #打印客户端的地址
            print("{}wrote".format(self.client_address[0]))
            #打印客户端发来的信息     
            print(self.data)
            #将客户端发来的信息变成大写病返回       
            self.request.sendall(self.data.upper())

if __name__ == "__main__":
    host, port = "127.0.0.1", 8008
    server = socketserver.TCPServer((host, port), MyTCPHandler)
    server.serve_forever()

多进程的情况
#!/usr/bin/env python3

import socketserver

class MyTCPHandler(socketserver.BaseRequestHandler):

    def handle(self):
        while True: #可以重复接收
            #等待客户端的链接，接收数据
            self.data = self.request.recv(1024).strip()
            #打印客户端的地址
            print("{}wrote".format(self.client_address[0]))
            #打印客户端发来的信息     
            print(self.data)
            #将客户端发来的信息变成大写病返回       
            self.request.sendall(self.data.upper())

if __name__ == "__main__":
    host, port = "127.0.0.1", 8008
    server = socketserver.ThreadingTCPServer((host, port), MyTCPHandler)
    server.serve_forever()

ThreadingTCPServer源码剖析

内部调用流程为：
启动服务端程序
执行 TCPServer.init 方法，创建服务端Socket对象并绑定 IP 和 端口
执行 BaseServer.init 方法，将自定义的继承自SocketServer.BaseRequestHandler 的类 MyRequestHandle赋值给 self.RequestHandlerClass
执行 BaseServer.server_forever 方法，While 循环一直监听是否有客户端请求到达 …
当客户端连接到达服务器
执行 ThreadingMixIn.process_request 方法，创建一个 “线程” 用来处理请求
执行 ThreadingMixIn.process_request_thread 方法
执行 BaseServer.finish_request 方法，执行 self.RequestHandlerClass() 即：执行 自定义 MyRequestHandler 的构造方法（自动调用基类BaseRequestHandler的构造方法，在该构造方法中又会调用 MyRequestHandler的handle方法）

ThreadingTCPServer相关源码：

BaseServer

class BaseServer:

    """Base class for server classes.

    Methods for the caller:

    - __init__(server_address, RequestHandlerClass)
    - serve_forever(poll_interval=0.5)
    - shutdown()
    - handle_request()  # if you do not use serve_forever()
    - fileno() -> int   # for select()

    Methods that may be overridden:

    - server_bind()
    - server_activate()
    - get_request() -> request, client_address
    - handle_timeout()
    - verify_request(request, client_address)
    - server_close()
    - process_request(request, client_address)
    - shutdown_request(request)
    - close_request(request)
    - handle_error()

    Methods for derived classes:

    - finish_request(request, client_address)

    Class variables that may be overridden by derived classes or
    instances:

    - timeout
    - address_family
    - socket_type
    - allow_reuse_address

    Instance variables:

    - RequestHandlerClass
    - socket

    """

    timeout = None

    def __init__(self, server_address, RequestHandlerClass):
        """Constructor.  May be extended, do not override."""
        self.server_address = server_address
        self.RequestHandlerClass = RequestHandlerClass
        self.__is_shut_down = threading.Event()
        self.__shutdown_request = False

    def server_activate(self):
        """Called by constructor to activate the server.

        May be overridden.

        """
        pass

    def serve_forever(self, poll_interval=0.5):
        """Handle one request at a time until shutdown.

        Polls for shutdown every poll_interval seconds. Ignores
        self.timeout. If you need to do periodic tasks, do them in
        another thread.
        """
        self.__is_shut_down.clear()
        try:
            while not self.__shutdown_request:
                # XXX: Consider using another file descriptor or
                # connecting to the socket to wake this up instead of
                # polling. Polling reduces our responsiveness to a
                # shutdown request and wastes cpu at all other times.
                r, w, e = _eintr_retry(select.select, [self], [], [],
                                       poll_interval)
                if self in r:
                    self._handle_request_noblock()
        finally:
            self.__shutdown_request = False
            self.__is_shut_down.set()

    def shutdown(self):
        """Stops the serve_forever loop.

        Blocks until the loop has finished. This must be called while
        serve_forever() is running in another thread, or it will
        deadlock.
        """
        self.__shutdown_request = True
        self.__is_shut_down.wait()

    # The distinction between handling, getting, processing and
    # finishing a request is fairly arbitrary.  Remember:
    #
    # - handle_request() is the top-level call.  It calls
    #   select, get_request(), verify_request() and process_request()
    # - get_request() is different for stream or datagram sockets
    # - process_request() is the place that may fork a new process
    #   or create a new thread to finish the request
    # - finish_request() instantiates the request handler class;
    #   this constructor will handle the request all by itself

    def handle_request(self):
        """Handle one request, possibly blocking.

        Respects self.timeout.
        """
        # Support people who used socket.settimeout() to escape
        # handle_request before self.timeout was available.
        timeout = self.socket.gettimeout()
        if timeout is None:
            timeout = self.timeout
        elif self.timeout is not None:
            timeout = min(timeout, self.timeout)
        fd_sets = _eintr_retry(select.select, [self], [], [], timeout)
        if not fd_sets[0]:
            self.handle_timeout()
            return
        self._handle_request_noblock()

    def _handle_request_noblock(self):
        """Handle one request, without blocking.

        I assume that select.select has returned that the socket is
        readable before this function was called, so there should be
        no risk of blocking in get_request().
        """
        try:
            request, client_address = self.get_request()
        except socket.error:
            return
        if self.verify_request(request, client_address):
            try:
                self.process_request(request, client_address)
            except:
                self.handle_error(request, client_address)
                self.shutdown_request(request)

    def handle_timeout(self):
        """Called if no new request arrives within self.timeout.

        Overridden by ForkingMixIn.
        """
        pass

    def verify_request(self, request, client_address):
        """Verify the request.  May be overridden.

        Return True if we should proceed with this request.

        """
        return True

    def process_request(self, request, client_address):
        """Call finish_request.

        Overridden by ForkingMixIn and ThreadingMixIn.

        """
        self.finish_request(request, client_address)
        self.shutdown_request(request)

    def server_close(self):
        """Called to clean-up the server.

        May be overridden.

        """
        pass

    def finish_request(self, request, client_address):
        """Finish one request by instantiating RequestHandlerClass."""
        self.RequestHandlerClass(request, client_address, self)

    def shutdown_request(self, request):
        """Called to shutdown and close an individual request."""
        self.close_request(request)

    def close_request(self, request):
        """Called to clean up an individual request."""
        pass

    def handle_error(self, request, client_address):
        """Handle an error gracefully.  May be overridden.

        The default is to print a traceback and continue.

        """
        print '-'*40
        print 'Exception happened during processing of request from',
        print client_address
        import traceback
        traceback.print_exc() # XXX But this goes to stderr!
        print '-'*40

TCPServer

class TCPServer(BaseServer):

    """Base class for various socket-based server classes.

    Defaults to synchronous IP stream (i.e., TCP).

    Methods for the caller:

    - __init__(server_address, RequestHandlerClass, bind_and_activate=True)
    - serve_forever(poll_interval=0.5)
    - shutdown()
    - handle_request()  # if you don't use serve_forever()
    - fileno() -> int   # for select()

    Methods that may be overridden:

    - server_bind()
    - server_activate()
    - get_request() -> request, client_address
    - handle_timeout()
    - verify_request(request, client_address)
    - process_request(request, client_address)
    - shutdown_request(request)
    - close_request(request)
    - handle_error()

    Methods for derived classes:

    - finish_request(request, client_address)

    Class variables that may be overridden by derived classes or
    instances:

    - timeout
    - address_family
    - socket_type
    - request_queue_size (only for stream sockets)
    - allow_reuse_address

    Instance variables:

    - server_address
    - RequestHandlerClass
    - socket

    """

    address_family = socket.AF_INET

    socket_type = socket.SOCK_STREAM

    request_queue_size = 5

    allow_reuse_address = False

    def __init__(self, server_address, RequestHandlerClass, bind_and_activate=True):
        """Constructor.  May be extended, do not override."""
        BaseServer.__init__(self, server_address, RequestHandlerClass)
        self.socket = socket.socket(self.address_family,
                                    self.socket_type)
        if bind_and_activate:
            try:
                self.server_bind()
                self.server_activate()
            except:
                self.server_close()
                raise

    def server_bind(self):
        """Called by constructor to bind the socket.

        May be overridden.

        """
        if self.allow_reuse_address:
            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.socket.bind(self.server_address)
        self.server_address = self.socket.getsockname()

    def server_activate(self):
        """Called by constructor to activate the server.

        May be overridden.

        """
        self.socket.listen(self.request_queue_size)

    def server_close(self):
        """Called to clean-up the server.

        May be overridden.

        """
        self.socket.close()

    def fileno(self):
        """Return socket file number.

        Interface required by select().

        """
        return self.socket.fileno()

    def get_request(self):
        """Get the request and client address from the socket.

        May be overridden.

        """
        return self.socket.accept()

    def shutdown_request(self, request):
        """Called to shutdown and close an individual request."""
        try:
            #explicitly shutdown.  socket.close() merely releases
            #the socket and waits for GC to perform the actual close.
            request.shutdown(socket.SHUT_WR)
        except socket.error:
            pass #some platforms may raise ENOTCONN here
        self.close_request(request)

    def close_request(self, request):
        """Called to clean up an individual request."""
        request.close()

ThreadingMixIn

class ThreadingMixIn:
    """Mix-in class to handle each request in a new thread."""

    # Decides how threads will act upon termination of the
    # main process
    daemon_threads = False

    def process_request_thread(self, request, client_address):
        """Same as in BaseServer but as a thread.

        In addition, exception handling is done here.

        """
        try:
            self.finish_request(request, client_address)
            self.shutdown_request(request)
        except:
            self.handle_error(request, client_address)
            self.shutdown_request(request)

    def process_request(self, request, client_address):
        """Start a new thread to process the request."""
        t = threading.Thread(target = self.process_request_thread,
                             args = (request, client_address))
        t.daemon = self.daemon_threads
        t.start()

ThreadingMixIn

ThreadingTCPServer

class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass

RequestHandler相关源码

class BaseRequestHandler:

    """Base class for request handler classes.

    This class is instantiated for each request to be handled.  The
    constructor sets the instance variables request, client_address
    and server, and then calls the handle() method.  To implement a
    specific service, all you need to do is to derive a class which
    defines a handle() method.

    The handle() method can find the request as self.request, the
    client address as self.client_address, and the server (in case it
    needs access to per-server information) as self.server.  Since a
    separate instance is created for each request, the handle() method
    can define arbitrary other instance variariables.

    """

    def __init__(self, request, client_address, server):
        self.request = request
        self.client_address = client_address
        self.server = server
        self.setup()
        try:
            self.handle()
        finally:
            self.finish()

    def setup(self):
        pass

    def handle(self):
        pass

    def finish(self):
        pass

SocketServer.BaseRequestHandler

ForkingTCPServer

ForkingTCPServer和ThreadingTCPServer的使用和执行流程基本一致，只不过在内部分别为请求者建立 “线程” 和 “进程”。
ForkingTCPServer和ThreadingTCPServer的区别，只是将实例中的

server = socketserver.ThreadingTCPServer((host, port), MyTCPHandler)

更改为

server = socketserver.ForkingTCPServer((host, port), MyTCPHandler)