Python之数据处理

---

靠别人不如靠自己，学学学学学学学学！

---

Tab替换为“,”

前言：因为是大数据实验室，导师要求节点数量要达到千万级，这种量级的数据用编辑器修改的话编辑器也要崩了，所以必须得用python处理了！

原数据

需求

• 将每一行的Tab替换为英文字符“,”
  代码

#-*- coding=utf-8 -*-

txtfile = "AMiner1.txt"
newtxtfile = open("new"+txtfile, "w")
with open(txtfile, 'r') as file_to_read:
    lines = file_to_read.readlines()
    for line in lines:
        line = line.replace("\t", ",")     # 将这一行的 "\t"(Tab)替换为 ","
        newtxtfile.write(line)             # 写入新的文件中
file_to_read.close()
newtxtfile.close()

结果

---

txt转csv

txt文件数据格式

代码

import pandas as pd
txtfile = "test.txt"
csvfile = "test.csv"

list = []
name = ['source','target','weight']  

with open(txtfile,'r') as file_to_read:
    lines = file_to_read.readlines()
    for line in lines:
        data = line.split()
        list.append(data)
    test = pd.DataFrame(columns=name, data=list)
    test.to_csv(csvfile)
file_to_read.close()

结果

---

删除某列中值为x的行

import pandas as pd
txtfile = "test.txt"
csvfile = "test.csv"

list = []
name = ['source','target','weight']

with open(txtfile,'r') as file_to_read:
    lines = file_to_read.readlines()
    for line in lines:
        data = line.split()
        if not data[2] is '1':                   # 就加了这么一行，data[2]指的是weight这一列，
        										 # 删除了weight值为1的行
            list.append(data)
    test = pd.DataFrame(columns=name, data=list)
    test.to_csv(csvfile)
file_to_read.close()

结果

---