爬虫学习——（一）Request库请求网页

目录

一、requests库常见方式请求网页

1.GET请求

基本实例

添加请求头

百度网页http请求实战

抓取网页数据

抓取二进制数据

2.POST请求

3.响应

---

在使用之前，确保安装requests库，可以在任务栏cmd里使用pip3来安装：

pip3 install requests

一、requests库常见方式请求网页

import requests

r = requests.get('https://www.httpbin.org/get')
r = requests.post('https://www.httpbin.org/post')
r = requests.put('https://www.httpbin.org/put')
r = requests.delete('https://www.httpbin.org/delete')
r = requests.patch('https://www.httpbin.org/patch')

1.GET请求

• 基本实例

import requests

r = requests.get('https://www.httpbin.org/get')
print(r.text)

运行结果：

{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate, br", 
    "Host": "www.httpbin.org", 
    "User-Agent": "python-requests/2.27.1", 
    "X-Amzn-Trace-Id": "Root=1-62d27f11-52cfa9120c9cb3430ff3acbf"
  }, 
  "origin": "120.238.232.103", 
  "url": "https://www.httpbin.org/get"
}

由上，返回结果包括args，headers(请求头)，IP，URL等信息。如果想要添加两个参数name和age，可以利用参数设置传递该信息，其中参数以字典的形式传入

import requests
data = {
    "name":"LZQ",
    "age" : 22
}
r = requests.get('https://www.httpbin.org/get',params=data)
print(r.text)

运行结果：

{
  "args": {
    "age": "22", 
    "name": "LZQ"
  }, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate, br", 
    "Host": "www.httpbin.org", 
    "User-Agent": "python-requests/2.27.1", 
    "X-Amzn-Trace-Id": "Root=1-62d28029-58d27395661203591b286a70"
  }, 
  "origin": "120.238.232.103", 
  "url": "https://www.httpbin.org/get?name=LZQ&age=22"
}

• 添加请求头

import requests

headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"}
r = requests.get('https://www.httpbin.org/get',headers = headers)
print(r.text)

运行结果：：

{
  "args": {}, 
  "headers": {