如何使用jq在Linux命令行上解析JSON文件

Fatmawati Achmad Zaenuri/Shutterstock Fatmawati Achmad Zaenuri / Shutterstock

JSON is one of the most popular formats for transferring text-based data around the web. It’s everywhere, and you’re bound to come across it. We’ll show you how to handle it from the Linux command line using the jq command.

JSON是用于在网络上传输基于文本的数据的最受欢迎的格式之一。 它无处不在，您一定会遇到它。 我们将向您展示如何使用jq命令从Linux命令行处理它。

JSON和jq (JSON and jq)

JSON stands for JavaScript Object Notation. It’s a scheme that allows data to be encoded into plain text files, in a self-describing way. There are no comments in a JSON file—the contents should be self-explanatory. Each data value has a text string called a “name” or “key.” This tells you what the data value is. Together, they’re known as name:value pairs, or key:value pairs. A colon (:) separates a key from its value.

JSON代表JavaScript Object Notation 。 这是一种允许以自描述方式将数据编码为纯文本文件的方案。 JSON文件中没有注释-内容应不言自明。 每个数据值都有一个称为“名称”或“键”的文本字符串。 这告诉您数据值是什么。 它们一起被称为“名称：值对”或“键：值对”。 冒号( : )分离它的值的密钥。

An “object” is a collection of key:value pairs. In a JSON file, an object begins with an open curly brace ({) and ends with a closing brace (}). JSON also supports “arrays,” which are ordered lists of values. An array begins with an opening bracket ([) and ends with a closing one (]).

“对象”是键：值对的集合。 在一个JSON文件，对象的开头是一个大括号( { )，并用一个闭括号结束( } )。 JSON还支持“数组”，它们是值的有序列表。 数组以左括号( [ )开始，以右括号( ]结束)。

From these simple definitions, of course, arbitrary complexity can arise. For example, objects can be nested within objects. Objects can contain arrays, and arrays can also contain objects. All of which can have open-ended levels of nesting.

当然，根据这些简单的定义，可能会出现任意的复杂性。 例如，对象可以嵌套在对象内。 对象可以包含数组，并且数组也可以包含对象。 所有这些都可以具有开放式的嵌套级别。

In practice, though, if the layout of JSON data is convoluted, the design of the data layout should probably use a rethink. Of course, if you’re not generating the JSON data, just trying to use it, you have no say in its layout. In those cases, unfortunately, you just have to deal with it.

但是，实际上，如果对JSON数据的布局进行了卷积，则数据布局的设计可能应该重新考虑。 当然，如果您不生成JSON数据，而只是尝试使用它，则在其布局中没有发言权。 不幸的是，在这些情况下，您只需要处理它即可。

Most programming languages have libraries or modules that allow them to parse JSON data. Sadly, the Bash shell has no such functionality.

大多数编程语言都具有允许它们解析JSON数据的库或模块。 可悲的是，Bash shell没有这种功能。

Necessity being the mother of invention, though, the jq utility was born! With jq, we can easily parse JSON in the Bash shell. And it doesn’t matter whether you have to work with well-engineered, elegant JSON, or the stuff nightmares are made of.

必需成为发明之母，但是jq实用程序诞生了！ 使用jq ，我们可以轻松地在Bash shell中解析JSON 。 而且，无论您是使用精心设计，优雅的JSON还是做噩梦的东西，都没有关系。

如何安装jq (How to Install jq)

We had to install jq on all the Linux distributions we used to research this article.

我们必须在用于研究本文的所有Linux发行版上安装jq 。

To install jq on Ubuntu type this command:

要在Ubuntu上安装jq ，请输入以下命令：

sudo apt-get install jq

To install jq on Fedora, type this command:

要在Fedora上安装jq ，请输入以下命令：

sudo dnf install jq

To install jq on Manjaro, type this command:

要在Manjaro上安装jq ，请键入以下命令：

sudo pacman -Sy jq

如何使JSON可读 (How to Make JSON Readable)

JSON doesn’t care about white space, and layout doesn’t affect it. As long as it follows the rules of JSON grammar, systems that process JSON can read and understood it. Because of this, JSON is often transmitted as a simple, long string, without any consideration of layout. This saves a bit of space because tabs, spaces, and new-line characters don’t have to be included in the JSON. Of course, the downside to all this is when a human tries to read it.

JSON不在乎空格，布局也不影响它。 只要遵循JSON语法规则，处理JSON的系统就可以读取和理解它。 因此，JSON通常作为简单的长字符串传输，而无需考虑布局。 这样可以节省一些空间，因为制表符，空格和换行符不必包含在JSON中。 当然，这一切的弊端是人们试图阅读它时。

Let’s pull a short JSON object from the NASA site that tells us the position of the International Space Station. We’ll use curl, which can download files to retrieve the JSON object for us.

让我们从NASA站点提取一个简短的JSON对象，该对象告诉我们国际空间站的位置。 我们将使用curl ，它可以下载文件来为我们检索JSON对象。

We don’t care about any of the status messages curl usually generates, so we’ll type the following, using the -s (silent) option:

我们不在乎curl通常生成的任何状态消息，因此我们将使用-s (静默)选项键入以下内容：

curl -s http://api.open-notify.org/iss-now.json

Now, with a bit of effort, you can read this. You have to pick out the data values, but it isn’t easy or convenient. Let’s repeat this, but this time we’ll pipe it through jq.

现在，您只需稍作努力，就可以阅读本文。 您必须选择数据值，但这并不容易。 让我们重复一遍，但是这次我们将其通过jq传递。

jq uses filters to parse JSON, and the simplest of these filters is a period (.), which means “print the entire object.” By default, jq pretty-prints the output.

jq使用过滤器来解析JSON，而这些过滤器中最简单的是一个句点( . )，这意味着“打印整个对象”。 默认情况下， jq漂亮打印输出。

We put it all together and type the following:

我们将所有内容放在一起，然后键入以下内容：

curl -s http://api.open-notify.org/iss-now.json | jq .

That’s much better! Now, we can see exactly what’s going on.

那好多了！ 现在，我们可以确切地看到发生了什么。

The entire object is wrapped in curly braces. It contains two key:name pairs: message and timestamp. It also contains an object called iss_position, which contains two key:value pairs: longitude and latitude.

整个对象都用大括号括起来。 它包含两个key：name对： message和timestamp 。 它还包含一个名为iss_position的对象，该对象包含两个key：value对： longitude和latitude 。

We’ll try this once more. This time we’ll type the following, and redirect the output into a file called “iss.json”:

我们将再次尝试。 这次我们将键入以下内容，并将输出重定向到名为“ iss.json”的文件中：

curl -s http://api.open-notify.org/iss-now.json | jq . > iss.json

cat iss.json

This gives us a well laid out copy of the JSON object on our hard drive.

这为我们在硬盘驱动器上提供了布局合理的JSON对象副本。

访问数据值 (Accessing Data Values)

As we saw above, jq can extract data values being piped through from JSON. It can also work with JSON stored in a file. We’re going to work with local files so the command line isn’t cluttered with curl commands. This should make it a bit easier to follow.

正如我们在上面看到的， jq可以提取从JSON传递过来的数据值。 它也可以使用存储在文件中的JSON。 我们将使用本地文件，以便命令行不会被curl命令弄乱。 这应该使它更容易理解。

The simplest way to extract data from a JSON file is to provide a key name to obtain its data value. Type a period and the key name without a space between them. This creates a filter from the key name. We also need to tell jq which JSON file to use.

从JSON文件提取数据的最简单方法是提供一个密钥名称以获得其数据值。 键入一个句点和键名称之间没有空格。 这将根据键名创建一个过滤器。 我们还需要告诉jq哪个JSON文件。

We type the following to retrieve the message value:

我们键入以下内容来检索message值：

jq .message iss.json

jq prints the text of the message value in the terminal window.

jq在终端窗口中输出message值的文本。

If you have a key name that includes spaces or punctuation, you have to wrap its filter in quotation marks. Care is usually taken to use characters, numbers, and underscores only so the JSON key names are not problematic.

如果您的键名包含空格或标点符号，则必须将其过滤器用引号引起来。 通常只注意使用字符，数字和下划线，因此JSON密钥名称没有问题。

First, we type the following to retrieve the timestamp value:

首先，我们键入以下内容以获取timestamp值：

jq .timestamp iss.json

The timestamp value is retrieved and printed in the terminal window.

时间戳值被检索并打印在终端窗口中。

But how can we access the values inside the iss_position object? We can use the JSON dot notation. We’ll include the iss_position object name in the “path” to the key value. To do this, the name of the object the key is inside will precede the name of the key itself.

但是，我们如何访问iss_position对象内部的值？ 我们可以使用JSON点表示法。 我们将在iss_position的“路径”中包含iss_position对象名称。 为此，键所在的对象的名称应位于键本身的名称之前。

We type the following, including the latitude key name (note there are no spaces between “.iss_position” and “.latitude”):

我们输入以下内容，包括latitude键名称(请注意，“。iss_position”和“ .latitude”之间没有空格)：

jq .iss_position.latitude iss.json

To extract multiple values, you have to do the following:

要提取多个值，您必须执行以下操作：

• List the key names on the command line.
  在命令行上列出键名。

• Separate them with commas (,).

  用逗号( , )分隔。

• Enclose them in quotation marks (") or apostrophes (').

  将其括在引号( " )或撇号( ' )中。

With that in mind, we type the following:

考虑到这一点，我们键入以下内容：

jq ".iss_position.latitude, .timestamp" iss.json

The two values print to the terminal window.

这两个值将打印到终端窗口。

处理数组 (Working with Arrays)

Let’s grab a different JSON object from NASA.

让我们从NASA获取另一个JSON对象。

This time, we’ll use a list of the astronauts who are in space right now:

这次，我们将使用现在在太空中的宇航员的清单：

curl -s http://api.open-notify.org/astros.json

Okay, that worked, so let’s do it again.

好的，那行得通，所以让我们再做一次。

We’ll type the following to pipe it through jq and redirect it to a file called “astro.json”:

我们将输入以下内容以将其通过jq传递，并将其重定向到名为“ astro.json”的文件：

curl -s http://api.open-notify.org/astros.json | jq . > astro.json

Now let’s type the following to check our file:

现在，让我们键入以下内容来检查我们的文件：

less astro.json

As shown below, we now see the list of astronauts in space, as well as their spacecrafts.

如下所示，我们现在看到太空中的宇航员及其航天器的列表。

This JSON object contains an array called people. We know it’s an array because of the opening bracket ([) (highlighted in the screenshot above). It’s an array of objects that each contain two key:value pairs:  name and craft.

此JSON对象包含一个名为people的数组。 我们知道这是一个数组，这是因为括号( [ )(在上面的屏幕快照中已突出显示)。 它是一个对象数组，每个对象包含两个key：value对： name和craft 。

Like we did earlier, we can use the JSON dot notation to access the values. We must also include the brackets ([]) in the name of the array.

像我们之前所做的那样，我们可以使用JSON点表示法来访问值。 我们还必须在数组名称中包含方括号( [] )。

With all that in mind, we type the following:

考虑到所有这些，我们键入以下内容：

jq ".people[].name" astro.json

This time, all the name values print to the terminal window. What we asked jq to do was print the name value for every object in the array. Pretty neat, huh?

这次，所有名称值都将打印到终端窗口。 我们让jq要做的是打印数组中每个对象的名称值。 很整洁吧？

We can retrieve the name of a single object if we put its position in the array in the brackets ([]) on the command line. The array uses zero-offset indexing, meaning the object in the first position of the array is zero.

如果将单个对象的名称放在命令行中方括号( [] )中的数组中，则可以检索单个对象的名称。 数组使用零偏移量索引，这意味着数组第一位置的对象为零。

To access the last object in the array you can use -1; to get the second to last object in the array, you can use -2, and so on.

要访问数组中的最后一个对象，可以使用-1; 要获得数组中倒数第二个对象，可以使用-2，依此类推。

Sometimes, the JSON object provides the number of elements in the array, which is the case with this one. Along with the array, it contains a key:name pair called number with a value of six.

有时，JSON对象提供数组中元素的数量，这种情况就是这种情况。 与数组一起，它包含一个名为number的key：name对，值为6。

The following number of objects are in this array:

此数组中包含以下对象：

jq ".people[1].name" astro.json

jq ".people[3].name" astro.json

jq ".people[-1].name" astro.json

jq ".people[-2].name" astro.json

You can also provide a start and end object within the array. This is called “slicing,” and it can be a little confusing. Remember the array uses a zero-offset.

您还可以在数组中提供一个开始和结束对象。 这称为“切片”，可能会有些混乱。 请记住，数组使用零偏移量。

To retrieve the objects from index position two, up to (but not including) the object at index position four, we type the following command:

要从索引位置2检索对象，直到(但不包括)索引位置4的对象，我们键入以下命令：

jq ".people[2:4]" astro.json

This prints the objects at array index two (the third object in the array) and three (the fourth object in the array). It stops processing at array index four, which is the fifth object in the array.

这将打印数组索引为2(数组中的第三个对象)和三个索引(数组中的第四个对象)的对象。 它将停止在数组索引4(数组的第五个对象)处进行处理。

The way to better understand this is to experiment on the command line. You’ll soon see how it works.

更好地理解这一点的方法是在命令行上进行实验。 您很快就会看到它是如何工作的。

如何使用带有过滤器的管道 (How to Use Pipes with Filters)

You can pipe the output from one filter to another, and you don’t have to learn a new symbol. The same as the Linux command line, jq uses the vertical bar (|) to represent a pipe.

您可以将输出从一个过滤器传递到另一个过滤器，而不必学习新的符号。 与Linux命令行相同， jq使用竖线( | )表示管道。

We’ll tell jq to pipe the people array into the .name filter, which should list the names of the astronauts in the terminal window.

我们将告诉jq将people数组通过管道传递到.name过滤器，该过滤器应在终端窗口中列出宇航员的姓名。

We type the following:

我们输入以下内容：

jq ".people[] | .name" astro.json

创建数组和修改结果 (Creating Arrays and Modifying Results)

We can use jq to create new objects, such as arrays. In this example, we’ll extract three values and create a new array that contains those values. Note the opening ([) and closing brackets (]) are also the first and last characters in the filter string.

我们可以使用jq创建新对象，例如数组。 在此示例中，我们将提取三个值并创建一个包含这些值的新数组。 请注意，开头( [ )和结尾方括号( ] )也是过滤字符串中的第一个和最后一个字符。

We type the following:

我们输入以下内容：

jq "[.iss-position.latitude, iss_position.longitude, .timestamp]" iss.json

The output is wrapped in brackets and separated by commas, making it a correctly formed array.

输出包装在方括号中，并用逗号分隔，从而使其形成正确的数组。

Numeric values can also be manipulated as they’re retrieved. Let’s pull the timestamp from the ISS position file, and then extract it again and change the value that’s returned.

检索数值时也可以对其进行操作。 让我们从ISS位置文件中提取timestamp ，然后再次提取它并更改返回的值。

To do so, we type the following:

为此，我们键入以下内容：

jq ".timestamp" iss.json

jq ".timestamp - 1570000000" iss.json

This is useful if you need to add or remove a standard offset from an array of values.

如果您需要从值数组中添加或删除标准偏移量，这将很有用。

Let’s type the following to remind ourselves what the iss.json file contains:

让我们键入以下内容来提醒自己iss.json文件包含什么：

jq . iss.json

Let’s say we want to get rid of the message key:value pair. It doesn’t have anything to do with the position of the International Space Station. It’s just a flag that indicates the location was retrieved successfully. If it’s surplus to requirements, we can dispense with it. (You could also just ignore it.)

假设我们要摆脱message key：value对。 它与国际空间站的位置无关。 它只是一个标志，指示位置已成功检索。 如果它超出了需求，我们可以省掉它。 (您也可以忽略它。)

We can use jq‘s delete function, del(), to delete a key:value pair. To delete the message key:value pair, we type this command:

我们可以使用jq的delete函数del()来删除一个key：value对。 要删除消息key：value对，我们输入以下命令：

jq "del(.message)" iss.json

Note this doesn’t actually delete it from the “iss.json” file; it just removes it from the output of the command. If you need to create a new file without the message key:value pair in it, run the command, and then redirect the output into a new file.

注意，这实际上并没有从“ iss.json”文件中删除它； 它只是将其从命令输出中删除。 如果需要创建一个不带message key：value对的新文件，请运行命令，然后将输出重定向到新文件中。

更加复杂的JSON对象 (More Complicated JSON Objects)

Let’s retrieve some more NASA data. This time, we’ll use a JSON object that contains information on meteor impact sites from around the world. This is a bigger file with a far more complicated JSON structure than those we’ve dealt with previously.

让我们检索更多NASA数据。 这次，我们将使用一个JSON对象，其中包含来自世界各地的流星影响站点的信息。 与我们之前处理过的文件相比，这是一个更大的文件，具有更复杂的JSON结构。

First, we’ll type the following to redirect it to a file called “strikes.json”:

首先，我们将键入以下内容以将其重定向到名为“ strikes.json”的文件：

curl -s https://data.nasa.gov/resource/y77d-th95.json | jq . > strikes.json

To see what JSON looks like, we type the following:

要查看JSON的外观，我们输入以下内容：

less strikes.json

As shown below, the file begins with an opening bracket ([), so the entire object is an array. The objects in the array are collections of key:value pairs, and there’s a nested object called geolocation. The geolocation object contains further key:value pairs, and an array called coordinates.

如下图所示，文件以左括号( [ )开头，因此整个对象是一个数组。 数组中的对象是key：value对的集合，并且有一个称为geolocation的嵌套对象。 geolocation对象还包含其他key：value对，以及一个称为coordinates的数组。

Let’s retrieve the names of the meteor strikes from the object at index position 995 through the end of the array.

让我们从数组结尾处的索引位置995的对象中检索流星撞击的名称。

We’ll type the following to pipe the JSON through three filters:

我们将键入以下内容以通过三个过滤器通过管道传递JSON：

jq ".[995:] | .[] | .name" strikes.json

The filters function in the following ways:

过滤器通过以下方式起作用：

• .[995:]: This tells jq to process the objects from array index 995 through the end of the array. No number after the colon ( : ) is what tells jq to continue to the end of the array.

  .[995:] ：这告诉jq处理从数组索引995到数组末尾的对象。 冒号后无号( : )是什么告诉jq继续到数组的末尾。

• .[]: This array iterator tells jq to process each object in the array.

  .[] ：此数组迭代器告诉jq处理数组中的每个对象。

• .name: This filter extracts the name value.

  .name ：此过滤器提取名称值。

With a slight change, we can extract the last 10 objects from the array. A “-10” instructs jq to start processing objects 10 back from the end of the array.

稍作更改，我们就可以从数组中提取最后10个对象。 “ -10”指示jq从数组末尾开始处理对象10。

We type the following:

我们输入以下内容：

jq ".[-10:] | .[] | .name" strikes.json

Just as we have in previous examples, we can type the following to select a single object:

就像前面的示例一样，我们可以键入以下内容来选择单个对象：

jq ".[650].name" strikes.json

We can also apply slicing to strings. To do so, we’ll type the following to request the first four characters of the name of the object at array index 234:

我们还可以对字符串应用切片。 为此，我们将键入以下内容以在数组索引234处请求对象名称的前四个字符：

jq ".[234].name[0:4]" strikes.json

We can also see a specific object in its entirety. To do this, we type the following and include an array index without any key:value filters:

我们还可以完整地看到一个特定的对象。 为此，我们键入以下内容并包括一个不带任何key：value过滤器的数组索引：

jq ".[234]" strikes.json

If you want to see only the values, you can do the same thing without the key names.

如果只想看这些值，则可以在没有键名的情况下做同样的事情。

For our example, we type this command:

对于我们的示例，我们键入以下命令：

jq ".[234][]" strikes.json

To retrieve multiple values from each object, we separate them with commas in the following command:

要从每个对象中检索多个值，我们在以下命令中用逗号分隔它们：

jq ".[450:455] | .[] | .name, .mass" strikes.json

If you want to retrieve nested values, you have to identify the objects that form the “path” to them.

如果要检索嵌套值，则必须标识构成它们的“路径”的对象。

For example, to reference the coordinates values, we have to include the all-encompassing array, the geolocation nested object, and the nested coordinates array, as shown below.

例如，要引用coordinates值，我们必须包括无所不包的数组， geolocation嵌套对象和嵌套coordinates数组，如下所示。

To see the coordinates values for the object at index position 121 of the array, we type the following command:

要查看对象在数组索引位置121的coordinates值，我们键入以下命令：

jq ".[121].geolocation.coordinates[]" strikes.json

长度功能 (The length Function)

The jq length function gives different metrics according to what it’s been applied, such as:

jq length函数根据应用的内容给出不同的指标，例如：

• Strings: The length of the string in bytes.

  字符串：字符串的长度(以字节为单位)。

• Objects: The number of key:value pairs in the object.

  对象：对象中键：值对的数量。

• Arrays: The number of array elements in the array.

  Arrays ：数组中数组元素的数量。

The following command returns the length of the name value in 10 of the objects in the JSON array, starting at index position 100:

以下命令从索引位置100开始，返回JSON数组中10个对象中name值的长度：

jq ".[100:110] | .[].name | length" strikes.json

To see how many key:value pairs are in the first object in the array, we type this command:

要查看数组的第一个对象中有多少键：值对，我们键入以下命令：

jq ".[0] | length" strikes.json

功能键 (The keys Function)

You can use the keys function to find out about the JSON you’ve got to work with. It can tell you what the names of the keys are, and how many objects there are in an array.

您可以使用keys函数来查找要使用的JSON。 它可以告诉您键的名称是什么，以及数组中有多少个对象。

To find the keys in the people object in the “astro.json” file, we type this command:

要在“ astro.json”文件中的people对象中找到键，请键入以下命令：

jq ".people.[0] | keys" astro.json

To see how many elements are in the people array, we type this command:

要查看people数组中有多少个元素，我们键入以下命令：

jq ".people | keys" astro.json

This shows there are six, zero-offset array elements, numbered zero to five.

这显示了六个零偏移的数组元素，编号为0到5。

has()函数 (The has() Function)

You can use the has() function to interrogate the JSON and see whether an object has a particular key name. Note the key name must be wrapped in quotation marks. We’ll wrap the filter command in single quotes ('), as follows:

您可以使用has()函数来查询JSON并查看对象是否具有特定的键名。 请注意，密钥名称必须用引号引起来。 我们将filter命令包装在单引号( ' )中，如下所示：

jq '.[] | has("nametype")' strikes.json

Each object in the array is checked, as shown below.

检查数组中的每个对象，如下所示。

If you want to check a specific object, you include its index position in the array filter, as follows:

如果要检查特定对象，请在数组过滤器中包括其索引位置，如下所示：

jq '.[678] | has("nametype")' strikes.json

没有它就不要靠近JSON (Don’t Go Near JSON Without It)

The jq utility is the perfect example of the professional, powerful, fast software that makes living in the Linux world such a pleasure.

jq实用程序是专业，强大，快速的软件的完美示例，它使生活在Linux世界中变得如此愉快。

This was just a brief introduction to the common functions of this command—there’s a whole lot more to it. Be sure to check out the comprehensive jq manual if you want to dig deeper.

这只是对该命令的常用功能的简要介绍，还有很多其他内容。 如果您想更深入地了解，请务必阅读全面的jq手册。

翻译自: https://www.howtogeek.com/529219/how-to-parse-json-files-on-the-linux-command-line-with-jq/