urllib

最新推荐文章于 2021-03-24 16:11:04 发布

路和远方

最新推荐文章于 2021-03-24 16:11:04 发布

阅读量134

点赞数 1

分类专栏： Python 文章标签： request

本文链接：https://blog.csdn.net/shuzhuchengfu/article/details/83623817

版权

Python 专栏收录该内容

34 篇文章 1 订阅

订阅专栏

这里写自定义目录标题

编解码

	from urllib import parse
    data = {"name":"张三","age":10}
    ue = parse.urlencode(data)
    print(ue)
    print(parse.parse_qs(ue))

结果

	name=%E5%BC%A0%E4%B8%89&age=10
	{'name': ['张三'], 'age': ['10']}

URL切割

    url = "https://www.baidu.com/user_info?username=张三&age=10"
    result = parse.urlsplit(url)
    print(result.scheme)
    print(result.netloc)
    print(result.path)
    print(result.query)

结果

https
www.baidu.com
/user_info
username=张三&age=10

GET请求

	from urllib import parse
	from urllib import request
    headers = {
        'User-Agent':'Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1'
    }
    # get请求
    # 方法一:
    resp = request.urlopen("http://www.baidu.com")
    print(resp.read())
    # 方法二:
    req = request.Request("http://www.baidu.com",headers=headers)
    resp = request.urlopen(req)
    print(resp.read())

POST 请求

	from urllib import parse
	from urllib import request
    # post 请求
    # 方法一:
    data = {"name":"张三"}
    data = parse.urlencode(data).encode(encoding='utf-8')
    resp = request.urlopen("http://www.baidu.com",data=data)
    print(resp.read())
    # 方法二:
    req = request.Request("http://www.baidu.com",headers=headers,data=data)
    resp = request.urlopen(req)
    print(resp.read())

下载文件

	from urllib import request
    request.urlretrieve("https://www.baidu.com/img/bd_logo1.png?where=super","baidu.jpg")

代理

    # 代理
    # 免费代理网站
    # http://www.xicidaili.com/
    handler = request.ProxyHandler({"http":"116.77.205.209:8118"})
    opener = request.build_opener(handler)
    req = request.Request("http://httpbin.org/ip")
    resp = opener.open(req)
    # 或者
    # resp = opener.open("http://httpbin.org/ip")
    print(resp.read())

Cookie

保存到本地

    cookiejar = MozillaCookieJar("cookie.txt")
    handler = request.HTTPCookieProcessor(cookiejar)
    opener = request.build_opener(handler)
    headers = {
        'User-Agent':'Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1'
    }
    req = request.Request("http://www.baidu.com",headers=headers)
    # opener
    resp = opener.open(req)
    print(resp.read())
    cookiejar.save(ignore_discard=True,ignore_expires=True)

加载本地

    cookiejar = MozillaCookieJar("cookie.txt")
    # 加载本地cookie
    cookiejar.load(ignore_expires=True,ignore_discard=True)
    handler = request.HTTPCookieProcessor(cookiejar)
    opener = request.build_opener(handler)
    headers = {
        'User-Agent':'Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1'
    }
    req = request.Request("http://www.baidu.com",headers=headers)
    # opener
    resp = opener.open(req)
    print(resp.read())