爬虫学习，request学习记录1

request学习记录

导入包

import requests

UA伪装

伪装的目的:告诉门户网站自己是个人

让爬虫对应的请求载体伪装为一个浏览器

if name == ‘main’:
headers = {
‘User-Agent’: ‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36’
}
# 1.指定url
kw = input(‘enter a word：’)
param = {
‘query’: kw
}
url = ‘https://www.sogou.com/web’

# 2.发请求
response = requests.get(url=url, params=param, headers=headers)

# 3.获取相应数据
page_text = response.text

# 4.持久化存储
filename = kw + '.html'
with open(filename,'w',encoding ='utf-8') as f:
    f.write(page_text)

print(kw,'保存好了')