jq-linux64,Linux json处理工具jq

官网: https://stedolan.github.io/jq/

jq 是一款命令行下处理 JSON 数据的工具。其可以接受标准输入，命令管道或者文件中的 JSON 数据，经过一系列的过滤器(filters)和表达式的转后形成我们需要的数据结构并将结果输出到标准输出中。jq 的这种特性使我们可以很容易地在 Shell 脚本中调用它。

安装JQ

wget -O jq https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64

chmod +x ./jq

mv jq /usr/bin

JQ的命令格式

jq [options] filter [files]

**options：**

--version：输出jq的版本信息并退出

--slurp/-s：读入整个输入流到一个数组。

--raw-input/-R：不作为JSON解析，将每一行的文本作为字符串输出到屏幕。

--null-input/ -n：不读取任何输入，过滤器运行使用null作为输入。一般用作从头构建JSON数据。

--compact-output /-c：使输出紧凑，而不是把每一个JSON对象输出在一行。

--colour-output / -C：打开颜色显示

--monochrome-output / -M：关闭颜色显示

--ascii-output /-a：指定输出格式为ASCII

-raw-output /-r ：如果过滤的结果是一个字符串，那么直接写到标准输出(去掉字符串的引号)

**filter：**

. : 默认输出

.foo: 输出指定属性，foo代表属性。

.[foo] ：输出指定数组元素。foo代表数组下标。

.[]：输出指定数组中全部元素

， ：指定多个属性作为过滤条件时，用逗号分隔

| ： 将指定的数组元素中的某个属性作为过滤条件

**files：**

JOSN格式文件。

JQ的使用

原始数据

# curl aaa.com/v1/catalog/nodes

[{ID:eb8c8e8f-4ae6-a24a-ec6b-d97f02fa62a1,Node:consul-0,Address:172.30.90.14,Datacenter:dc1,TaggedAddresses:{lan:172.30.90.14,wan:172.30.90.14},Meta:{consul-network-segment:},CreateIndex:7,ModifyIndex:8},{ID:b1e19b9d-a062-6569-b97d-4134de9dec65,Node:consul-1,Address:172.30.70.5,Datacenter:dc1,TaggedAddresses:{lan:172.30.70.5,wan:172.30.70.5},Meta:{consul-network-segment:},CreateIndex:5,ModifyIndex:10},{ID:0ea4808b-f3bf-6337-b084-f35de1e550e4,Node:consul-2,Address:172.30.22.12,Datacenter:dc1,TaggedAddresses:{lan:172.30.22.12,wan:172.30.22.12},Meta:{consul-network-segment:},CreateIndex:6,ModifyIndex:9}]

格式化输出

[root@k8sMaster01 delete-consul-critical]# curl aaa.com/v1/catalog/nodes | jq

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

100 716 100 716 0 0 43923 0 --:--:-- --:--:-- --:--:-- 44750

[

{

"ID": "eb8c8e8f-4ae6-a24a-ec6b-d97f02fa62a1",

"Node": "consul-0",

"Address": "172.30.90.14",

"Datacenter": "dc1",

"TaggedAddresses": {

"lan": "172.30.90.14",

"wan": "172.30.90.14"

},

"Meta": {

"consul-network-segment": ""

},

"CreateIndex": 7,

"ModifyIndex": 8

},

{

"ID": "b1e19b9d-a062-6569-b97d-4134de9dec65",

"Node": "consul-1",

"Address": "172.30.70.5",

"Datacenter": "dc1",

"TaggedAddresses": {

"lan": "172.30.70.5",

"wan": "172.30.70.5"

},

"Meta": {

"consul-network-segment": ""

},

"CreateIndex": 5,

"ModifyIndex": 10

},

{

"ID": "0ea4808b-f3bf-6337-b084-f35de1e550e4",

"Node": "consul-2",

"Address": "172.30.22.12",

"Datacenter": "dc1",

"TaggedAddresses": {

"lan": "172.30.22.12",

"wan": "172.30.22.12"

},

"Meta": {

"consul-network-segment": ""

},

"CreateIndex": 6,

"ModifyIndex": 9

}

]

获取某字段的值

# 是带引号的值

[root@k8sMaster01 delete-consul-critical]# curl -s aaa.com/v1/catalog/nodes | jq '.[].Node'

"consul-0"

"consul-1"

"consul-2"

# 把处理后的结果当做普通行(-r)

[root@k8sMaster01 delete-consul-critical]# curl -s aaa.com/v1/catalog/nodes | jq -r '.[].Node'

consul-0

consul-1

consul-2

# 获取指定段中某字段的值,并输出为1行

[root@k8sMaster01 delete-consul-critical]# curl -s http://aaa.com/v1/catalog/nodes | jq -r '.[1].Node,.[1].Address' | xargs echo

consul-1 172.30.70.5

提取指定字段形成新的json

# 修改前的json串(-c参数)

[root@k8sMaster01 delete-consul-critical]# curl -s aaa.com/v1/catalog/nodes | jq -r '.[1]'

{

"ID": "b1e19b9d-a062-6569-b97d-4134de9dec65",

"Node": "consul-1",

"Address": "172.30.70.5",

"Datacenter": "dc1",

"TaggedAddresses": {

"lan": "172.30.70.5",

"wan": "172.30.70.5"

},

"Meta": {

"consul-network-segment": ""

},

"CreateIndex": 5,

"ModifyIndex": 10

}

# 修改后的json串,吧Node修改为NodeName,吧Address修改为ConsulIPAddress

[root@k8sMaster01 delete-consul-critical]# curl -s aaa.com/v1/catalog/nodes | jq -c '{NodeName:.[1].Node,ConsulIpAddress:.[1].Address}'

{"NodeName":"consul-1","ConsulIpAddress":"172.30.70.5"}

csv文件转json串

[root@k8sMaster01 tmp]# cat jq.csv

baiyongjie,beijing,24

houmm,beijing,23

xiaoming,hebei,18

[root@k8sMaster01 tmp]# jq -R -c 'split(",")|{"Name":.[0],"City":.[1],"age":.[2]|tonumber}' jq.csv

{"Name":"baiyongjie","City":"beijing","age":24}

{"Name":"houmm","City":"beijing","age":23}

{"Name":"xiaoming","City":"hebei","age":18}

json串转csv文件

[root@k8sMaster01 tmp]# cat jq.json

{"Name":"baiyongjie","City":"beijing","age":24}

{"Name":"houmm","City":"beijing","age":23}

{"Name":"xiaoming","City":"hebei","age":18}

[root@k8sMaster01 tmp]# jq -r "[.Name,.City,.age]|@csv" jq.json

"baiyongjie","beijing",24

"houmm","beijing",23

"xiaoming","hebei",18