python爬虫入门(4)
本次学习简单的多线程
from flask import Flask
import time
app = Flask(__name__)
@app.route('/')
def index():
time.sleep(3)
return 'welcome!'
if __name__ == '__main__':
app.run(threaded = True)
上述为经典的试验网址
下面是多线程的基本操作:
import requests
import time
from threading import Thread
start_t = time.time()
url = 'http://127.0.0.1:5000/'
for i in range(10):
resp = requests.get(url)
u = resp.text
print(u)
end_t = time.time()
m = end_t - start_t
print('访问10次,共:',m)
import requests
start_t = time.time()
url = 'http://127.0.0.1:5000/'
def get_html(url):
resp = requests.get(url)
print(resp.text)
#10线程
th_list = []
for i in range(10):
th = Thread(target=get_html,args=(url,)) #形式为元组
th.start()
th_list.append(th)
for th in th_list:
th.join()
end_t = time.time()
print('time:',end_t - start_t)
比较上下两个会发现时间骤减,达到了多线程省时的效果。
注意建立服务器的代码需要和爬虫代码分别在两个文件中运行