socket python 收 发 队列 线程_Python套接字接收/发送多线程

I am writing a Python program where in the main thread I am continuously (in a loop) receiving data through a TCP socket, using the recv function. In a callback function, I am sending data through the same socket, using the sendall function. What triggers the callback is irrelevant. I've set my socket to blocking.

My question is, is this safe to do? My understanding is that a callback function is called on a separate thread (not the main thread). Is the Python socket object thread-safe? From my research, I've been getting conflicting answers.

解决方案

Sockets in Python are not thread safe.

You're trying to solve a few problems at once:

Sockets are not thread-safe.

recv is blocking and blocks the main thread.

sendall is being used from a different thread.

You may solve these by either using asyncio or solving it the way asyncio solves it internally: By using select.select together with a socketpair, and using a queue for the incoming data.

import select

import socket

import queue

# Any data received by this queue will be sent

send_queue = queue.Queue()

# Any data sent to ssock shows up on rsock

rsock, ssock = socket.socketpair()

main_socket = socket.socket()

# Create the connection with main_socket, fill this up with your code

# Your callback thread

def different_thread():

# Put the data to send inside the queue

send_queue.put(data)

# Trigger the main thread by sending data to ssock which goes to rsock

ssock.send(b"\x00")

# Run the callback thread

while True:

# When either main_socket has data or rsock has data, select.select will return

rlist, _, _ = select.select([main_socket, rsock], [], [])

for ready_socket in rlist:

if ready_socket is main_socket:

data = main_socket.recv(1024)

# Do stuff with data, fill this up with your code

else:

# Ready_socket is rsock

rsock.recv(1) # Dump the ready mark

# Send the data.

main_socket.sendall(send_queue.get())

We use multiple constructs in here. You will have to fill up the empty spaces with your code of choice. As for the explanation:

We first create a send_queue which is a queue of data to send. Then, we create a pair of connected sockets (socketpair()). We need this later on in order to wake up the main thread as we don't wish recv() to block and prevent writing to the socket.

Then, we connect the main_socket and start the callback thread. Now here's the magic:

In the main thread, we use select.select to know if the rsock or main_socket has any data. If one of them has data, the main thread wakes up.

Upon adding data to the queue, we wake up the main thread by signaling ssock which wakes up rsock and thus returns from select.select.

In order to fully understand this, you'll have to read select.select(), socketpair() and queue.Queue().