python 线程与urllib操作

最新推荐文章于 2022-05-02 01:50:18 发布

冒雨前行的蜗牛

最新推荐文章于 2022-05-02 01:50:18 发布

阅读量519

点赞数

分类专栏：笔记文章标签： python 大数据

本文链接：https://blog.csdn.net/zengshuihai/article/details/124046033

版权

笔记专栏收录该内容

37 篇文章 1 订阅

订阅专栏

本文介绍了Python中多线程的同步使用，通过线程锁实现线程间的有序执行，并展示了线程优先级队列的工作原理，如Queue的get和put方法。同时，还演示了使用urllib进行HTTP请求，获取网页头部信息的基本操作。

摘要由CSDN通过智能技术生成

1.多线程同步:

import threading ,time

exitFlag = 0 

class  My_thread(threading.Thread):
   def __init__(self,threadID,name,delay):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.delay = delay
   def fun(self):  
       print('开启线程:' + self.name)
        #获取锁,用语线程同步
       threadLock.acquire()
       print_time(self.name,self.delay,3)
        #释放锁,开启下一个线程
       threadLock.release()
def print_time(threadname,delay,counter):
    while counter:
        if exitFlag:
            threadname.exit()
        time.sleep(delay)
        print(('%s:%s') %(threadname,time.ctime(time.time())))
        counter -= 1
threadLock = threading.Lock()
threads = []
#创建新线程:
thread1 = Mythread(1,'Thread -1',1)
thread2 = Mythread(2,'Thread -2',2)
#开启新线程:
thread1.start()
thread2.start()
#添加线程到线程列表:
threads.append(thread1)
threads.append(thread2)
#等待所有线程完成:
for t in threads:
    t.join()
print('退出主线程')
输出:
开启线程:Thread -1
开启线程:Thread -2
Thread -1:Fri Apr  8 17:16:19 2022
Thread -1:Fri Apr  8 17:16:21 2022
Thread -1:Fri Apr  8 17:16:22 2022
Thread -2:Fri Apr  8 17:16:24 2022
Thread -2:Fri Apr  8 17:16:26 2022
Thread -2:Fri Apr  8 17:16:28 2022
退出主线程

2.线程优先列队(Queue):
Queue.Qzise():返回队列大小;
Queue.emply():如果队列为空返回True,反之为False;
Queue.full():如果队列满了，返回True,反之False;
Queue.full 与 maxsize 大小对应
Queue.get([block[, timeout]])获取队列，timeout等待时间
Queue.get_nowait() 相当Queue.get(False)
Queue.put(item) 写入队列，timeout等待时间
Queue.put_nowait(item) 相当Queue.put(item, False)
Queue.task_done() 在完成一项工作之后，Queue.task_done()函数向任务已经完成的队列发送一个信号
Queue.join() 实际上意味着等到队列为空，再执行别的操作
摘自:https://www.runoob.com/python3/python3-tutorial.html

import requests ,time , queue
exitFlag = 0
class My_thread(threading.Thread):
    def __init__(self,ThreadID,name,q):
        threading.Thread.__init__(self)
        self.ThreadID = ThreadID
        self.name= name
        self.q = q
    def fun(self):
        print('开启线程:', self.name)
        process_data(self.name,self.q)
        print('开启线程:', self.name)
def process_data(threadname,q):
    queueLock.acquire()
    if not workqueue.empy():
        data = q.get()
        queueLock.release()
        print('%s process %s' %(threadname , data))
    else:
        queueLock.release()
    time.sleep(1)
threadList = ["Thread-1", "Thread-2", "Thread-3"]
nameList = ["One", "Two", "Three", "Four", "Five"]
queueLock = threading.Lock()
workQueue = queue.Queue(10)
threads = []
threadID = 1

# 创建新线程
for tName in threadList:
    thread = My_thread(threadID, tName, workQueue)
    thread.start()
    threads.append(thread)
    threadID += 1
    
输出:
开启线程：Thread-1
开启线程：Thread-2
开启线程：Thread-3
Thread-3 processing One
Thread-1 processing Two
Thread-2 processing Three
Thread-3 processing Four
Thread-1 processing Five
退出线程：Thread-3
退出线程：Thread-2
退出线程：Thread-1
退出主线程

3.urllib操作:

from urllib.request import urlopen

myURL = urlopen("https://www.runoob.com/")
print(myURL.getheaders())
输出:
[('Server', 'JSP3/2.0.14'), ('Date', 'Fri, 08 Apr 2022 10:15:14 GMT'), ('Content-Type', 'text/html; charset=UTF-8'), ('Transfer-Encoding', 'chunked'), ('Connection', 'close'), ('Age', '69430'), ('Accept-Ranges', 'bytes'), ('Vary', 'Accept-Encoding'), ('Link', '<http://www.runoob.com/wp-json/>; rel="https://api.w.org/"'), ('Timing-Allow-Origin', '*'), ('Ohc-Cache-HIT', 'zs2ct62 [2], njctcache138 [1], czix138 [1]'), ('Ohc-File-Size', '320211')]