URL解析:
urlparse
就是将网址按规律分段;
1 from urllib.parse import urlparse
2
3 result = urlparse('http://www.baidu.com/index.html;user?id=5#comment')
4 print(type(result), result)
结果是:
<class 'urllib.parse.ParseResult'> ParseResult(scheme='http', netloc='www.baidu.com', path='/index.html', params='user', query='id=5', fragment='comment')
url可以分成六个字段;
里边两个网址不同
1 from urllib.parse import urlparse
2
3 result = urlparse('http://www.baidu.com/index.html;user?id=5#comment', scheme='https')
4 print( result)
1 from urllib.parse import urlparse
2
3 result = urlparse('www.baidu.com/index.html;user?id=5#comment', scheme='https')
4 print( result)
向前拼接:将fragments所带的字段向前拼接
1 from urllib.parse import urlparse
2
3 result = urlparse('http://baidu.com/index.html;user?id=5#comment', allow_fragments=False)
4 print( result)
1 from urllib.parse import urlparse
2
3 result = urlparse('http://baidu.com/index.html#comment', allow_fragments=False)
4 print( result)
拼接链接:就是将各个字段拼接成一个链接;
1 from urllib.parse import urlunparse
2
3 data = ['http', 'www.baidu.com', 'index.html', 'user', 'a=6', 'comment']
4 result = urlunparse(data)
5 print(result)
拼接url:
urljoin方法;
1 from urllib.parse import urljoin
2
3 print(urljoin('http://www.baidu.com', 'FAQ.html'))
4 print(urljoin('http://www.baidu.com', 'https://cuiqingcai.com/FAQ.html'))
5 print(urljoin('http://www.baidu.com/about.html', 'https://cuiqingcai.com/FAQ.html'))
6 print(urljoin('http://www.baidu.com/about.html', 'https://cuiqingcai.com/FAQ.html?question=2'))
7 print(urljoin('http://www.baidu.com?wd=abc', 'https://cuiqingcai.com/index.php'))
8 print(urljoin('http://www.baidu.com', '?category=2#comment'))
9 print(urljoin('http://www.baidu.com#comment', '?category=2#comment'))
结果:
http://www.baidu.com/FAQ.html
https://cuiqingcai.com/FAQ.html
https://cuiqingcai.com/FAQ.html
https://cuiqingcai.com/FAQ.html?question=2
https://cuiqingcai.com/index.php
http://www.baidu.com?category=2#comment
http://www.baidu.com?category=2#comment
把列表里的内容加到链接里:
1 from urllib.parse import urlencode
2
3 params = {'name':'germey', 'age':'22'}
4 base_url = 'http://www.baidu.com'
5 url = base_url + urlencode(params)
6 print(url)
结果是: