json处理工具jq

@ 2018-03-01

jq工具可以检查JSON的合法性，并把JSON格式化成更友好更可读的格式。

将一团乱麻的JSON格式化成个更可读的形式，原始json文件jsonTest数据如下：

{"_location":"(32.121,  45.262)","_ip":"2.12.1.211","_action":"click button","_uid":"694666ee-8918-48f8-bb92-0a756a3f1f31","_timestamp":"1496999163163"}
{"_location":"(32.121,  45.262)","_ip":"7.12.1.211","_action":"click button","_uid":"868be529-f59e-4f16-8ea5-08c4612ede9a","_timestamp":"1496999164165"}
{"_location":"(32.121,  45.262)","_ip":"2.12.1.211","_action":"click button","_uid":"49b63380-2a5a-453d-aa5b-57e74cd1a8d7","_timestamp":"1496999165170"}
{"_location":"(32.121,  45.262)","_ip":"3.12.1.211","_action":"click button","_uid":"60fd093a-9f6a-4716-81c6-72a5f07d3b4f","_timestamp":"1496999166171"}
{"_location":"(32.121,  45.262)","_ip":"3.12.1.211","_action":"click button","_uid":"056cf558-56b1-4312-aba1-0b0eb71ba78e","_timestamp":"1496999167172"}
{"_location":"(32.121,  45.262)","_ip":"8.12.1.211","_action":"click button","_uid":"a2ecd34c-5458-43a2-b851-89eb6b3985ae","_timestamp":"1496999168174"}

使用cat jsonTest | jq . 之后的json数据格式如下：

{
  "_timestamp": "1496999163163",
  "_uid": "694666ee-8918-48f8-bb92-0a756a3f1f31",
  "_action": "click button",
  "_ip": "2.12.1.211",
  "_location": "(32.121,  45.262)"
}
{
  "_timestamp": "1496999164165",
  "_uid": "868be529-f59e-4f16-8ea5-08c4612ede9a",
  "_action": "click button",
  "_ip": "7.12.1.211",
  "_location": "(32.121,  45.262)"
}
......

有时需要将格式化的json串作为一行输出，如Hive中的json必须是一行一个json串，如json文件info.json的原始数据如下：

{
        "generated_on": "2017-12-03 08:41:42.057563",
        "slice": "0-999",
        "version": "v1"
}

使用cat info.json | jq -c .可以得到以下数据：

{"version":"v1","slice":"0-999","generated_on":"2017-12-03 08:41:42.057563"}

jq的操作文档可参考https://stedolan.github.io/jq/manual/