Python3爬取新闻并存入MySQL
Python3爬取新闻并存入MySQL
项目内容:
1.爬取某新闻正文
2.将新闻正文存入Mysql
本文以两篇人民网新闻为例
一、 爬取新闻正文
# -*- coding:utf-8 -*-
import lxml.html
import tushare as ts
from sphinx.util import requests
import pymysql
pymysql.install_as_MySQLdb()
import mysqldbda
from sqlalchemy import create_engine
def reptile(web,xpath1,xpath2):
selector = lxml.html.fromstring(requests.get(web).content.decode('GBK'))
site = selector.xpath(xpath1)
selector_html = []
for i in range(len(</