python出租车数据_Python处理JSON格式数据（出租车轨迹数据）

1.主要是先通过观察发现JSON数据的规律，我们可以使用Notepad++工具查看JSON数据的结构，了解数据的层次。

即使文件不是真正的json格式，例如有可能是多个json格式放在一起，每一行是一个json文件。

例如

{json1}

{json2}

……

2.很大的文本我们可以先用python读取前500或前1000个字符来查看JSON格式都有什么

当我们了解了结构之后，可以先按行读取文件。

fname=open("m.txt","r")

print(fname.read(1000))

处理json格式，我们需要json和pandas库

例如我们的文件名为m.txt

内容大致为：(实际每一行不一样)

{ "_id" : "729c3318ad3b4d53b7a92a31010fd085" , "fid" : "BJJYJ_095994" , "y" : "2012" , "m" : "03" , "d" : "01" , "sp" : 728 , "sl" : 235650.0 , "st1" : 40534.0 , "st2" : 40534.0 , "bb" : "116.2164536,39.8669052,116.6208801,40.0528488" , "pts" : [ { "1" : 0 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.0 , "5" : 0.0 , "6" : "2012-03-01 06:28:28" , "7" : "2012-03-01 06:28:33" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 1 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.06486486486486487 , "5" : 0.0 , "6" : "2012-03-01 06:29:36" , "7" : "2012-03-01 06:29:41" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 2 , "2" : 39.9216499 , "3" : 116.6060028 , "4" : 1.894736842105263 , "5" : 0.0 , "6" : "2012-03-01 06:31:27" , "7" : "2012-03-01 06:31:34" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 3 , "2" : 39.9216805 , "3" : 116.6056519 , "4" : 16.494545454545456 , "5" : 23.0 , "6" : "2012-03-01 06:32:24" , "7" : "2012-03-01 06:32:30" , "8" : "0" , "9" : 354.0 , "10" : 1} ]}

{ "_id" : "729c3318ad3b4d53b7a92a31010fd085" , "fid" : "BJJYJ_095994" , "y" : "2012" , "m" : "03" , "d" : "01" , "sp" : 728 , "sl" : 235650.0 , "st1" : 40534.0 , "st2" : 40534.0 , "bb" : "116.2164536,39.8669052,116.6208801,40.0528488" , "pts" : [ { "1" : 0 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.0 , "5" : 0.0 , "6" : "2012-03-01 06:28:28" , "7" : "2012-03-01 06:28:33" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 1 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.06486486486486487 , "5" : 0.0 , "6" : "2012-03-01 06:29:36" , "7" : "2012-03-01 06:29:41" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 2 , "2" : 39.9216499 , "3" : 116.6060028 , "4" : 1.894736842105263 , "5" : 0.0 , "6" : "2012-03-01 06:31:27" , "7" : "2012-03-01 06:31:34" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 3 , "2" : 39.9216805 , "3" : 116.6056519 , "4" : 16.494545454545456 , "5" : 23.0 , "6" : "2012-03-01 06:32:24" , "7" : "2012-03-01 06:32:30" , "8" : "0" , "9" : 354.0 , "10" : 1} ]}

则处理代码如下

import json

import pandas as pd

j=0

with open(r'm.txt') as file:

for line in file:

m=json.loads(line)

n = m["pts"]

data = pd.DataFrame(n)

filename = "%d.csv"%j

f = open(filename , 'w')

j=j+1

data.to_csv(filename)

f.close()

处理的结果如图

每个文件的内容

,1,10,2,3,4,5,6,7,8,9

0,0,1,39.9216461,116.6059799,0.0,0.0,2012-03-01 06:28:28,2012-03-01 06:28:33,0,0.0

1,1,1,39.9216461,116.6059799,0.06486486486486487,0.0,2012-03-01 06:29:36,2012-03-01 06:29:41,0,0.0

2,2,1,39.9216499,116.6060028,1.894736842105263,0.0,2012-03-01 06:31:27,2012-03-01 06:31:34,0,0.0

3,3,1,39.9216805,116.6056519,16.494545454545456,23.0,2012-03-01 06:32:24,2012-03-01 06:32:30,0,354.0

4,4,1,39.9234695,116.6038361,24.588679245283018,23.0,2012-03-01 06:33:19,2012-03-01 06:33:24,0,302.0

5,5,1,39.9262543,116.6060333,16.599999999999998,21.0,2012-03-01 06:34:12,2012-03-01 06:34:18,0,36.0

这样json文件就被处理成可以使用的csv文件了，可以用于ArcGIS后期分析处理了