1.主要是先通过观察发现JSON数据的规律,我们可以使用Notepad++工具查看JSON数据的结构,了解数据的层次。
即使文件不是真正的json格式,例如有可能是多个json格式放在一起,每一行是一个json文件。
例如
{json1}
{json2}
……
2.很大的文本我们可以先用python读取前500或前1000个字符来查看JSON格式都有什么
当我们了解了结构之后,可以先按行读取文件。
fname=open("m.txt","r")
print(fname.read(1000))
处理json格式,我们需要json和pandas库
例如我们的文件名为m.txt
内容大致为:(实际每一行不一样)
{ "_id" : "729c3318ad3b4d53b7a92a31010fd085" , "fid" : "BJJYJ_095994" , "y" : "2012" , "m" : "03" , "d" : "01" , "sp" : 728 , "sl" : 235650.0 , "st1" : 40534.0 , "st2" : 40534.0 , "bb" : "116.2164536,39.8669052,116.6208801,40.0528488" , "pts" : [ { "1" : 0 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.0 , "5" : 0.0 , "6" : "2012-03-01 06:28:28" , "7" : "2012-03-01 06:28:33" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 1 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.06486486486486487 , "5" : 0.0 , "6" : "2012-03-01 06:29:36" , "7" : "2012-03-01 06:29:41" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 2 , "2" : 39.9216499 , "3" : 116.6060028 , "4" : 1.894736842105263 , "5" : 0.0 , "6" : "2012-03-01 06:31:27" , "7" : "2012-03-01 06:31:34" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 3 , "2" : 39.9216805 , "3" : 116.6056519 , "4" : 16.494545454545456 , "5" : 23.0 , "6" : "2012-03-01 06:32:24" , "7" : "2012-03-01 06:32:30" , "8" : "0" , "9" : 354.0 , "10" : 1} ]}
{ "_id" : "729c3318ad3b4d53b7a92a31010fd085" , "fid" : "BJJYJ_095994" , "y" : "2012" , "m" : "03" , "d" : "01" , "sp" : 728 , "sl" : 235650.0 , "st1" : 40534.0 , "st2" : 40534.0 , "bb" : "116.2164536,39.8669052,116.6208801,40.0528488" , "pts" : [ { "1" : 0 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.0 , "5" : 0.0 , "6" : "2012-03-01 06:28:28" , "7" : "2012-03-01 06:28:33" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 1 , "2" : 39.9216461 , "3" : 116.6059799 , "4" : 0.06486486486486487 , "5" : 0.0 , "6" : "2012-03-01 06:29:36" , "7" : "2012-03-01 06:29:41" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 2 , "2" : 39.9216499 , "3" : 116.6060028 , "4" : 1.894736842105263 , "5" : 0.0 , "6" : "2012-03-01 06:31:27" , "7" : "2012-03-01 06:31:34" , "8" : "0" , "9" : 0.0 , "10" : 1} , { "1" : 3 , "2" : 39.9216805 , "3" : 116.6056519 , "4" : 16.494545454545456 , "5" : 23.0 , "6" : "2012-03-01 06:32:24" , "7" : "2012-03-01 06:32:30" , "8" : "0" , "9" : 354.0 , "10" : 1} ]}
则处理代码如下
import json
import pandas as pd
j=0
with open(r'm.txt') as file:
for line in file:
m=json.loads(line)
n = m["pts"]
data = pd.DataFrame(n)
filename = "%d.csv"%j
f = open(filename , 'w')
j=j+1
data.to_csv(filename)
f.close()
处理的结果如图
每个文件的内容
,1,10,2,3,4,5,6,7,8,9
0,0,1,39.9216461,116.6059799,0.0,0.0,2012-03-01 06:28:28,2012-03-01 06:28:33,0,0.0
1,1,1,39.9216461,116.6059799,0.06486486486486487,0.0,2012-03-01 06:29:36,2012-03-01 06:29:41,0,0.0
2,2,1,39.9216499,116.6060028,1.894736842105263,0.0,2012-03-01 06:31:27,2012-03-01 06:31:34,0,0.0
3,3,1,39.9216805,116.6056519,16.494545454545456,23.0,2012-03-01 06:32:24,2012-03-01 06:32:30,0,354.0
4,4,1,39.9234695,116.6038361,24.588679245283018,23.0,2012-03-01 06:33:19,2012-03-01 06:33:24,0,302.0
5,5,1,39.9262543,116.6060333,16.599999999999998,21.0,2012-03-01 06:34:12,2012-03-01 06:34:18,0,36.0
这样json文件就被处理成可以使用的csv文件了,可以用于ArcGIS后期分析处理了