python3 读取大文件分解成若干小文件

有个数据实在太大了，有1.7G，打开慢，改文件也慢，我们将其分解成若干个中等文件

#!/usr/bin/env python3

# -*- coding: utf-8 -*-

f = open("123.sql",'r',encoding='utf-8')

readlist = []

i = 0

i=i+1

filename = "mytest_{0}.sql".format(i)

wf=open(filename, 'w', encoding='utf-8')

linecount=0

while 1:

    try:

        line = f.readline()

        wf.write(line)

        linecount=linecount+1       

        if(linecount >;= 200000):

            linecount = 0

            print(filename)

            i=i+1

            wf.close()

            filename = "mytest_{0}.sql".format(i)

            wf=open(filename, 'w', encoding='utf-8')                

        if not line:

            break        

    except Exception as e:

        print("read except:" + str(e))

        continue

        #print("read except:" + str(e))

f.close()

print(filename)

wf.close()        

有一个问题，之前的sql文件保存为utf-8格式，实际上在文件中夹杂中部分乱码，读取时会出错，所以需要try屏蔽

转载于:https://www.cnblogs.com/luhouxiang/p/4940056.html