下面展示一些 内联代码片
。
实例1
从京东https://item.jd.com/11198917385.html爬取信息
import requests
url="https://item.jd.com/11198917385.html"
try:
kv = {'user-agent':'Mozilla/5.0'}
r = requests.get(url,headers = kv)
r.raise_for_status()
r.encoding=r.apparent_encoding
return r.text
except:
return "产生异常"
这时候出现了问题:Non-UTF-8 code starting with ‘\xb2’ in file requests.py on line 9, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
解决办法:python默认的编码是UTF-8,汉字注释时需要在代码开头加上
#_*_ coding:gb2312_*_
结果还是出现问题SyntaxError: ‘return’ outside function
出现return函数只能用在函数里面
#_*_ coding:gb2312_*_
import requests
def GetHTMLText(url):
try:
kv = {'user-agent':'Mozilla/5.0'}
r = requests.get(url,headers = kv)
r.raise_for_status()
r.encoding=r.apparent_encoding
return r.text
except:
return "产生异常"
if __name__ =="__main__":
url="https://item.jd.com/11198917385.html"
print(GetHTMLText(url))
这时候还是产生异常
AttributeError: module ‘requests’ has no attribute ‘get’
因为我文件名取为requests,冲突了,改名之后就可以了
‘user-agent’:'Mozilla/5.0’将身份标志成浏览器,默认user-agent为爬虫
实例2
百度关键词接口:http://www.baidu.com/s?wd=keyword
360关键词接口:http://www.so.com/s?q=keyword
def GetHTMLText(url):
try:
kw={'wd':keyword}
kv={'user-agent':'Mozilla/5.0'}
r = requests.get(url,headers = kv,params=kw)
print(r.request.url)
r.raise_for_status()
r.encoding=r.apparent_encoding
print(len(r.text))
except:
return "产生异常"
if __name__ =="__main__":
url="https://www.baidu.com"
keyword="Python"
实例3
网络图片链接格式:http://www.example.com/picture.jpg
import requests
import os
url="http://www.example.com/picture.jpg"
root="D://pics//"
path=root+url.split('/')[-1]
try:
if not os.path.exists(root):
os.mkdir(root)
if not os.path.exisrs(path):
r=requests.get(url)
with open(path,'wb') as f:
f.write(r.content)
f.close()
print("文件保存成功")
else:
print("文件已存在")
except:
print("爬取失败")
实例4:IP地址归属地查询
import requests
url="http://m.ip138.com/ip.asp?ip="
try:
r=requests.get(url+'202.204.80.112')
r.raise_for_status()
r.encoding=r.apparent_encoding
print(r.text[-500:])
except:
print("爬取失败")