python爬取百度搜索答案题目和摘要

最新推荐文章于 2024-07-29 21:31:04 发布

小帝仙

最新推荐文章于 2024-07-29 21:31:04 发布

阅读量3.8k

点赞数

分类专栏：编程文章标签： python 百度搜索 url

本文链接：https://blog.csdn.net/u014737266/article/details/70056140

版权

编程专栏收录该内容

1 篇文章 0 订阅

订阅专栏

url就自行构造吧

# coding:utf-8
import urllib2
import re
from bs4 import BeautifulSoup

url = 'http://www.baidu.com/s?ie=utf-8&f=8&rsv_bp=1&rsv_idx=1&tn=baidu&wd=johnkey&oq=john&rsv_pq=88bbfd770000beed&rsv_t=be24xj7KYq9tbjeRa7Fu10sW1dFF0GNZI1%2FW31Bq8OsZWZIwSpuRZxdcfQo&rqlang=cn&rsv_enter=1&inputT=787&rsv_sug3=12&rsv_sug1=7&rsv_sug7=100&rsv_sug2=0&rsv_sug4=787'

request = urllib2.Request(url)
request.add_header('User-Agent','Mozilla/5.0')
response = urllib2.urlopen(request)

html = response.read()

soup = BeautifulSoup(html,'html.parser',from_encoding='utf-8')
links = soup.find_all('div',id=re.compile(r'\d+'))
for link in links:
print link.name,link['id'],link.get_text()