JING0981的博客

import requestsfrom bs4 import BeautifulSoupimport chardetimport randomimport timedef getHtml(url):#准备一个网址url=‘http://www.baidu.com’#获取响应网页response=requests.get(url)#获取编码code=chardet.detect...

#!/usr/bin/env python# -- coding: utf-8 -from gethtml import getHtmlfrom urllib import requestfrom bs4 import BeautifulSoupimport relv=[]for i in range(108):print(‘正在...

#!/usr/bin/env python-- coding: utf-8 -import csvimport pymysqlconn = pymysql.connect(host=‘localhost’, user=‘root’, password=‘1234’, db=‘danke’, charset=“utf8”)cursor = conn.cursor()with open(‘...

xpath提取网页内容 selector模块XML 节点选择表达式 描述nodename 选取此节点的所有子节点。/ 从根节点选取。// 从匹配选择的当前节点选择文档中的节点，而不考虑它们的位置。. 选取当前节点。… 选取当前节点的父节点。@ 选取属性。bookstore 选取 bookstore 元素的所有子节点。/bookstore 选取根元素 bookstore。boo...

beautifulsoup的使用 bs4模块准备代码信息，用来练习获取内容:from bs4 import BeautifulSoup#准备代码信息，用来练习获取内容html =’’’The Dormouse's story 123456 The Dormouse's story aaaaa The Dormouse's storya Once u...

准备头部和代理user_agent = [ #准备头部，列表“Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50”,“Mozilla/5.0 (Windows; U; Windows NT 6.1;...

写入mysql#!/usr/bin/env python-- coding: utf-8 -import pymysqlconn = pymysql.connect(host=‘IP’,user=‘用户名’,password=‘密码’,db=‘数据库’,charset=“utf8”)cursor = conn.cursor()#建立游标cursor当前的程序到数据之间连接管道sql =...

import re‘’’re.compile(string[, flags])re.match(pattern, string[, flags])re.search(pattern, string[, flags])re.split(pattern, string[, maxsplit])re.findall(pattern, string[, flags])re.finditer(...

import urllib2import urllib1、抓取网页response = urllib2.urlopen(“http://www.baidu.com/”)print response.read()‘’’分析：response = urllib2.urlopen(“http://www.baidu.com/”)调用urllib2的urlopen方法，传入一个url，这个...