新年礼物(寒假csv数据处理任务)

一个简单的数据处理任务

转眼间,寒假已经过半,前三周的任务大家都完成了吗?正所谓“寒假与你错过的知识终究会在组内考核时重逢”,上一次开组会很多同学反映看书的时候什么都懂,一到自己写代码的时候就忧心忡忡。为了解决大家日益增长的知识需求和止步不前的学习成果之间的矛盾,同时也是为了丰富大家充(wu)实(liao)的业余生活,现特别推出新年大礼包,给大家布置一个简单的数据处理任务。

时间安排

Begin: 2021-2-10 (Wed.) CST
End: 2021-2-21 (Sun.) CST

任务描述

本次任务要处理的数据共101227行,样例如下:

18 Jogging 102271561469000 -13.53 16.89 -6.4
18 Jogging 102271641608000 -5.75 16.89 -0.46
18 Jogging 102271681617000 -2.18 16.32 11.07
18 Jogging 3.36
18 Downstairs 103260201636000 -4.44 7.06 1.95
18 Downstairs 103260241614000 -3.87 7.55 3.3
18 Downstairs 103260321693000 -4.06 8.08 4.79
18 Downstairs 103260365577000 -6.32 8.66 4.94
18 Downstairs 103260403083000 -5.37 11.22 3.06
18 Downstairs 103260443305000 -5.79 9.92 2.53
6 Walking 0 0 0 3.214402

—>相应文件

Step 1

将数据集中所有信息异常的行删除。
比如上面的样例中第4行数据只有3个元素,而其他行都有6个元素,所以第4行是信息异常的行,将其删除。再如第12行数据的第3个元素明显也是有问题的,所以它也是信息异常的行,将其删除。
数据集中可能还会存在一些其他异常。
将全部信息处理之后,每行的元素以逗号为分隔符,写入文件test1
文件test1共100471行,样例如下:

6,Walking,23445542281000,-0.72,9.62,0.14982383
6,Walking,23445592299000,-4.02,11.03,3.445948
6,Walking,23470662276000,0.95,14.71,3.636633
...

Step 2

统计文件test1的数据中所有动作的数目并打印到屏幕,然后将动作数目对100取整后写入test2文件,多余的信息行抛弃。比如统计出Jogging的数量为3021次,则在屏幕上打印Movement: Jogging Amount: 3021,然后将前3000行信息写入test2文件。
文件test2共100200行。

Step 3

读取文件test2的数据,取每行的后3列元素,以空格为分隔符写入文件test3
文件test3共100200行,样例如下:

-0.72 9.62 0.14982383
-4.02 11.03 3.445948
0.95 14.71 3.636633
...

Step 4

读取文件test3的数据,每行数据为一组,每组组内的元素以空格为分隔符,组与组之间的数据以逗号为分隔符,每20组元素为一行,写入文件finally
文件finally共5010行,样例如下:

-0.72 9.62 0.14982383,-4.02 11.03 3.445948,0.95 14.71 3.636633,-3.57 5.75 -5.407278,-5.28 8.85 -9.615966,-1.14 15.02 -3.8681788,7.86 11.22 -1.879608,6.28 4.9 -2.3018389,0.95 7.06 -3.445948,-1.61 9.7 0.23154591,6.44 12.18 -0.7627395,5.83 12.07 -0.53119355,7.21 12.41 0.3405087,6.17 12.53 -6.701211,-1.08 17.54 -6.701211,-1.69 16.78 3.214402,-2.3 8.12 -3.486809,-2.91 0 -4.7535014,-2.91 0 -4.7535014,-4.44 1.84 -2.8330324

验收内容

  • 4个*.py文件

    • test1.py
    • test2.py
    • test3.py
    • finally.py
  • 4个运行Python脚本后生成的文件

    • test1
    • test2
    • test3
    • finally

注:如果对本次任务有疑问,请及时询问19级的学长和学姐。
祝大家春节快乐!

我的处理如下

test1.py

import csv
import sys
input_file = sys.argv[1]
output_file = sys.argv[2]
with open(input_file,'r',newline='') as csv_in_file:
    with open(output_file,'w',newline='') as csv_out_file:
        filereader = csv.reader(csv_in_file, delimiter=' ')
        filewriter = csv.writer(csv_out_file, delimiter=',')
        for row_list in filereader:
        	length = len(row_list)
        	if length == 6 and int(row_list[2]) != 0:
        		filewriter.writerow(row_list)

把异常数据进行处理

test2.py

import csv
import sys

Walking_1 = 0
Jogging_1 = 0
Upstairs_1 =0
Downstairs_1 =0
Standing_1 =0
Sitting_1 =0

Walking_2 = 0
Jogging_2 = 0
Upstairs_2 =0
Downstairs_2 =0
Standing_2 =0
Sitting_2 =0

input_file = sys.argv[1]
output_file = sys.argv[2]

with open(input_file, 'r', newline='') as csv_in_file:
    with open(output_file, 'w', newline='') as csv_out_file:
        filereader = csv.reader(csv_in_file)
        filewriter = csv.writer(csv_out_file)
        for row_list in filereader:
            if row_list[1] == 'Walking':
                Walking_1+=1
            if row_list[1] == 'Jogging':
                Jogging_1+=1
            if row_list[1] == 'Upstairs':
                Upstairs_1+=1
            if row_list[1] == 'Downstairs':
                Downstairs_1+=1
            if row_list[1] == 'Standing':
                Standing_1+=1
            if row_list[1] == 'Sitting':
                Sitting_1+=1

        print('Movement:Walking Amount:')
        print(Walking_1)
        print('Movement:Jogging Amount:')
        print(Jogging_1)
        print('Movement:Upstairs Amount:')
        print(Upstairs_1)
        print('Movement:Downstairs Amount:')
        print(Downstairs_1)
        print('Movement:Standing Amount:')
        print(Standing_1)
        print('Movement:Sitting Amount:')
        print(Sitting_1)

        Walking_1 = Walking_1 - Walking_1%100
        Jogging_1 = Jogging_1 - Jogging_1%100
        Upstairs_1 = Upstairs_1 - Upstairs_1%100
        Downstairs_1 = Downstairs_1 - Downstairs_1%100
        Standing_1 = Standing_1 - Standing_1%100
        Sitting_1 = Sitting_1 - Sitting_1%100
        print(Sitting_1)
        print(Jogging_1)
        print(Upstairs_1)
        print(Downstairs_1)    
        print(Standing_1)
        print(Walking_1)
        
with open(input_file, 'r', newline='') as csv_in_file:
    with open(output_file, 'w', newline='') as csv_out_file:

        filereader = csv.reader(csv_in_file)
        filewriter = csv.writer(csv_out_file)

        for row_list in filereader:
            if row_list[1] == 'Jogging' and Jogging_2< Jogging_1:
                Jogging_2 +=1
                filewriter.writerow(row_list)
            if row_list[1] == 'Upstairs' and Upstairs_2< Upstairs_1:
                Upstairs_2 +=1
                filewriter.writerow(row_list)
            if row_list[1] == 'Downstairs' and Downstairs_2 < Downstairs_1:
                Downstairs_2 +=1
                filewriter.writerow(row_list)
            if row_list[1] == 'Standing' and Standing_2 < Standing_1:
                Standing_2 +=1
                filewriter.writerow(row_list)
            if row_list[1] == 'Sitting' and Sitting_2 < Sitting_1:
                Sitting_2 +=1
                filewriter.writerow(row_list)
            if row_list[1] == 'Walking' and Walking_2 < Walking_1:
                Walking_2 +=1
                print(Downstairs_2)   
                filewriter.writerow(row_list)

        
           

test3.py

import csv
import sys
input_file = sys.argv[1]
output_file = sys.argv[2]
with open(input_file, 'r', newline='') as csv_in_file:
    with open(output_file, 'w', newline='') as csv_out_file:
        filereader = csv.reader(csv_in_file, delimiter=',')
        filewriter = csv.writer(csv_out_file, delimiter=' ')
        for row_list in filereader:
            filewriter.writerow(row_list[3:6])

test4.py

import csv
import sys

input_file = sys.argv[1]
output_file = sys.argv[2]

with open(input_file, 'r', newline='') as csv_in_file:
    with open(output_file, 'w', newline='') as csv_out_file:
        filereader = csv.reader(csv_in_file)
        filewriter = csv.writer(csv_out_file)
        total_date= []
        new_list = []
        new_row = []
        i = 1
        for row_list in filereader:
            total_date.append(row_list[0])
        for data in total_date:
            if i != 20:
                i += 1
                new_list.append(data)
            else:
                new_list.append(data)
                new_row.append(new_list)
                i = 1
                new_list = []
        for row in new_row:
            filewriter.writerow(row)
                        
     
           

常见的问题是处理列表与字符串形式的转换

评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值