Adapted from The Ultimate Interactive JQ Guide by Ishan Das Sharma (CC BY-NC-SA 4.0).

jq

Search, query, and modify JSON data with jq

jq is a tool for extracting and manipulating JSON, one of the most common plaintext data formats on the internet. jq can help:

1. Extract data from websites
2. Manage software configuration files
3. Monitor and analyze server logs

All examples are fully interactive, so feel free to modify the input and program code!

Basic operations

Selecting values

Everything in jq is a filter. The dot . is used to select the current object or element, and we can put the property name after it to access a key from an object:

Filtering arrays

The .[] notation is used to iterate over the elements of an array in a JSON document. It allows you to access each element of an array and perform operations on them.

The select() function is used to filter JSON data based on a specified condition or criteria. It is a powerful tool for extracting specific elements from a JSON document that meet certain conditions.

Similiar to shell scripting, jq works on a pipes-and-filters manner. We use the | to send the data from one filter to the next.

Indexing into arrays

To select a particular value from an array, use .[index] notation. Negative indices work count from the end of the array.

Slicing arrays

To select a range of values from an array, use .[start:end] notation (excluding the end value).

Combining filters

The pipe operator | can be used to chain as many filters or functions as we want:

Splitting strings

We can use the split() function to a split a string on a particular separator character. Note also the usage of .[0] to select the first index from the split array.

Mapping arrays

We can use the map function to run any operation on every element of the array and return a new array containing the outputs of that operation:

Handling null values

Null values can often mess up logic in our scripts, so we can filter them all out using map and select

Multiple outputs

Curly braces create a new object, which we can use for multiple outputs:

Nested objects

JSON is very commonly used to store nested objects, and we often need to traverse or manipulate such structures. jq gives us all the tools we need to make it easy:

Recursive descent

We can use .. to recursively descend through a tree of an object.

Filtering nested arrays

Statistical operations

jq is handy for conducting quick statistical analyses. Here’s most of the common operations related to that

Sorting arrays

Sorting an array is a basic operation that is useful for many statistics.

Unique array values

Extracting unique values from an array is another common operation.

Calculating averages

Calculating the mean or average of a dataset is a common statistical operation.

Grouping and aggregating

We can group an array of objects by a particular key and get an aggregated value of the other keys.