python爬取同花顺_网络爬虫页面解析的疑问？同花顺-问财

http://stock.10jqka.com.cn/company.shtml

搜了下汽车

25条信息

直接用爬虫爬这个网址，http://www.iwencai.com/stockpick/search?ts=1&f=1&qs=ops_ths_data&tid=stockpick&w=%E6%B1%BD%E8%BD%A6

一复制网址就变了。

http://www.iwencai.com/stockpick/search?ts=1&f=1&qs=ops_ths_data&tid=stockpick&w=%E6%B1%BD%E8%BD%A6

网址也是对的，为抓取不到。大家可以试下。

我用来抓京东的代码。

#!/usr/bin/python

# -*- coding: UTF-8 -*-

import requests

from bs4 import BeautifulSoup

from lxml import etree

rul = 'http://www.iwencai.com/stockpick/search?ts=1&f=1&qs=ops_ths_data&tid=stockpick&w=%E6%B1%BD%E8%BD%A6'

headers = {

'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36'

}

html.encoding = 'utf-8'

html = requests.get(rul)

soup = BeautifulSoup(html.text,'html.parser')

print(soup)

#name=soup.select('.cate_menu_item')

#for i in name:

#a=i.select('a')[0].text

#print(a)

#print(html.text)

#soup = etree.HTML(html.text)

#con_list = soup.xpath('.//*[@id="tableWrap"]/div[2]/div/div[2]/div/table/tbody/tr/td[4]/div')

#for i in con_list:

#name=i.xpath('/a/text()')

#name

一抓只抓出来个这个