python3.4.3下逐行读入txt文本并去重

逐行读取TXT文档并去重

读写文件时应注意的问题包括：

1.字符编码
2.操作完成即时关闭文件描述符
3.代码兼容性

几种方法：

#!/bin/python3

original_list1=[" "]
original_list2=[" "]
original_list3=[" "]
original_list4=[" "]
newlist1=[" "]
newlist2=[" "]
newlist3=[" "]
newlist4=[" "]
newtxt1=""
newtxt2=""
newtxt3=""
newtxt4=""

#first way to readline
f = open("duplicate_txt.txt","r+")       # 返回一个文件对象   
line = f.readline()              		 # 调用文件的 readline()方法  
while line:   
	original_list1.append(line)                   
	line = f.readline()    
f.close() 

#use "set()" remove duplicate str in the list
# in this way,list will sort randomly
newlist1 = list(set(