为什么有些xpath绝对路径拿不到数据_我写的Xpath 为什么爬取不到内容

–– coding:utf-8 ––

import lxml,requests,sys

from bs4 import BeautifulSoup

from lxml import etree

reload(sys)

sys.setdefaultencoding(“utf-8”)

def main():

url = 'https://image.baidu.com/search/flip?tn=baiduimage&ie=utf-8&word=%E6%9A%B4%E8%B5%B0%E6%BC%AB%E7%94%BB&pn=0'

req = requests.get(url).content

# soup = BeautifulSoup(req.content,’lxml’)

# imgs = soup.find_all(‘img’)

content = etree.HTML(req)

paths = content.xpath('//*[@id="imgid"]/ul/li[1]/a/img/text()')

# for img in imgs:

#

#     print img

# for img in imgs :

print paths

main()

在写爬虫的时候，使用xpath一定要确认一下网页的源代码中是否有数据，如果没有，说明是异步加载的

1. 浏览器输入这个连接即可看源代码，ctrl+f 查找imgid所在的位置

view-source:https://image.baidu.co