python3 pandas读取文本写入MySQL

import pymysql
import pandas as pd
from sqlalchemy import create_engine
pymysql.install_as_MySQLdb()

name = ['id','create_time','ruuid','version','addition',
        'bookname','content','dversion','grade','message','operatedate',
        'pageindex','realname','school','source','status','time','type','week','userid','caption']

data = pd.read_table('/Users/yangyang/Desktop/report_table.txt',error_bad_lines=False,names = name)
list = set()
dict = {}

for i,row in data.iterrows():
    label = "{} {} {} {} {} {} {}".format(row['time'],row['userid'],row['bookname'],row['type'],
                                    row['content'],row['message'],row['addition'])
    if label in list:
        if row['id'] > dict['id']:
            # data.drop(data.index[i])
            data= data.drop(i)
        else :
            dict['id'] = row['id']
    else:
        list.add(label)
        dict['id'] = row['id']

connect = create_engine('mysql+mysqldb://root:chineseall@192.168.1.177:3306/bigdata?charset=utf8mb4')
data.to_sql('report_table_clean',con=connect,if_exists='replace',index=False)
print('success')
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值