json

json online player:

jq online player:

reference:

basic

syntax for jq

SYNTAXDESCRIPTION

,

Filters separated by a comma will produce multiple independent outputs

?

Will ignores error if the type is unexpected

[]

Array construction

{}

Object construction

+

Concatenate or Add

-

Difference of sets or Substract

length

Size of selected element

⎮

Pipes are used to chain commands in a similar fashion than bash

dealing with json objects

DESCRIPTIONCOMMAND

Display all keys

jq 'keys'

Adds + 1 to all items

jq 'map_values(.+1)'

Delete a key

jq 'del(.foo)'

Convert an object to array

to_entries ⎮ map([.key, .value])

slicing and filtering

[!NOTE|label:references:]

  • select by type: with type been arrays, objects, iterables, booleans, numbers, normals, finites, strings, nulls, values, scalars

DESCRIPTIONCOMMAND

All

jq .[]

First

jq '.[0]'

Range

jq '.[2:4]'

First 3

jq '.[:3]'

Last 2

jq '.[-2:]'

Before Last

jq '.[-2]'

split

jq '.[] ⎮ split("/")[1]'

Select array of int by value

jq 'map(select(. >= 2))'

Select array of objects by value

jq '.[] ⎮ select(.id == "second")'

Select by type

jq '.[] ⎮ numbers'

mapping and transforming

DESCRIPTIONCOMMAND

Add + 1 to all items

jq 'map(.+1)'

Delete 2 items

jq 'del(.[1, 2])'

Concatenate arrays

jq 'add'

Flatten an array

jq 'flatten'

Create a range of numbers

jq '[range(2;4)]'

Display the type of each item

jq 'map(type)'

Sort an array of basic type

jq 'sort'

Sort an array of objects

jq 'sort_by(.foo)'

Group by a key - opposite to flatten

jq 'group_by(.foo)'

Minimun value of an array

jq 'min' See also min, max, min_by(path_exp), max_by(path_exp)

Remove duplicates

jq 'unique' or jq 'unique_by(.foo)' or jq 'unique_by(length)'

Reverse an array

jq 'reverse'

join

[!TIP] references:

$ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | join(":")'
thing:like

  • or

    $ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | "\(.[0]) \(.[1])"'
thing like

  • or

    $ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | "\(.[0]): \(.[1])"'
thing: like

  • or with reduce

    $ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | reduce .[1:][] as $i ("\(.[0])"; . + ",\($i)")'
thing,like

  • or

    $ echo '{ "k1": "v1", "k2": "v2", "k3": "v3", "k4": "v4" }' |
       jq -r '[.k1, .k2, .k3] | reduce .[1:][] as $i ("\(.[0])"; . + ",\($i)")'
v1,v2,v3

  • or with .first and .last

    $ echo '{ "users": [ { "first": "Stevie", "last": "Wonder" }, { "first": "Michael", "last": "Jackson" } ] }' |
       jq -r '.users[] | .first + " " + .last'
Stevie Wonder
Michael Jackson

  • or

    $ echo '{ "users": [ { "first": "Stevie", "last": "Wonder" }, { "first": "Michael", "last": "Jackson" } ] }' |
       jq -r '.users[] | .first + " " + (.last|tostring)'

  • or

    $ echo '{ "users": [ { "first": "Stevie", "last": "Wonder", "number": 1 }, { "first": "Michael", "last": "Jackson", "number": 2 } ] }' |
       jq -r '.users[] | .first + " " + (.number|tostring)'
Stevie 1
Michael 2

split

[!TIP] references:

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri'
/1
/2
/3

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri | split("/")[1]'
1
2
3

  • try online

    $ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq '.[].uri | split("/")[]'
""
"1"
""
"2"
""
"3"
$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri | split("/")'
[
  "",
  "1"
]
[
  "",
  "2"
]
[
  "",
  "3"
]

replacing

[!TIP|label:references:]

  • Replacing substrings in a string

  • sub(regex; tostring), sub(regex; string; flags)

    • Emit the string obtained by replacing the first match of regex in the input string with tostring, after interpolation. tostring should be a jq string, and may contain references to named captures. The named captures are, in effect, presented as a JSON object (as constructed by capture) to tostring, so a reference to a captured variable named "x" would take the form: "(.x)".

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri'
/1
/2
/3

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri | sub("/"; "")'
1
2
3

builtin operators

[!NOTE|label:references:]

debug

  • without debug

    $ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "SECOND", "val": 3}]'''  |
       jq '.[] | select( .val == (2, 3) )'

  • with debug:

    $ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "SECOND", "val": 3}]'''  |
       jq '.[] | select( .val == (2, 3) | debug )'
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",true]
{
  "id": "second",
  "val": 2
}
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",true]
{
  "id": "SECOND",
  "val": 3
}

select

[!NOTE|label:refrences:]

$ echo "[1,5,3,0,7]" |
       jq 'map(select(. >= 2))'
[
  5,
  3,
  7
]

  • or

    $ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2}]''' |
       jq '.[] | select(.id == "second")'
{
  "id": "second",
  "val": 2
}

  • select with any

    [!NOTE|label:references:]

    $ cat sample.json
[
  {
    "attributes": {
      "created": "2021-10-18T12:02:39+00:00",
      "enabled": true,
      "expires": null,
      "notBefore": null
    },
    "contentType": null,
    "id": "https://kjkljk./secrets/-/1",
    "managed": null,
    "name": "pw",
    "tags": {}
  },
  {
    "attributes": {
      "created": "2021-10-18T12:06:16+00:00",
      "enabled": false,
      "expires": null,
      "notBefore": null
    },
    "contentType": "",
    "id": "https://kjklj./secrets/-/2",
    "managed": null,
    "name": "pw",
    "tags": {}
  }
]

# get [{.id}] format
$ cat sample.json |
      jq -r 'map(select(any(.attributes; .enabled)) | {id})'
[
  {
    "id": "https://kjkljk./secrets/-/1"
  }
]

# get [.id] format
$ cat sample.json | jq -r 'map(select(any(.attributes; .enabled)) | .id)'
[
  "https://kjkljk./secrets/-/1"
]

# get .id format ( without `map()` )
$ cat sample.json | jq -r '.[] | select(any(.attributes; .enabled)) | .id'
https://kjkljk./secrets/-/1

# re-format
$ cat sample.json | jq -r '{"ids": .[] | select(any(.attributes; .enabled)) | .id}'
{
  "ids": "https://kjkljk./secrets/-/1"
}

contains

[!NOTE|label:reference:]

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "second-one", "val": 3}]''' |
       jq '.[] | select( .id | contains("second") )'
{
  "id": "second",
  "val": 2
}
{
  "id": "second-one",
  "val": 3
}

  • by using test

    $ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "second.one", "val": 3}]'''  |
       jq '.[] | select( .id | test("sec*") )'
{
  "id": "second",
  "val": 2
}
{
  "id": "second.one",
  "val": 3
}

inside

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "second-one", "val": 3}]'''  |
       jq '.[] | select( [.val] | inside([2,3]) )'
{
  "id": "second",
  "val": 2
}
{
  "id": "second-one",
  "val": 3
}

  • or

    $ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "SECOND", "val": 3}]'''  |
       jq '.[] | select( .val == (2, 3) )'
{
  "id": "second",
  "val": 2
}
{
  "id": "SECOND",
  "val": 3
}

toUpperCase : ascii_upcase

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "SECOND", "val": 3}]'''  |
       jq '.[] | select(.id | ascii_upcase == "SECOND")'
{
  "id": "second",
  "val": 2
}
{
  "id": "SECOND",
  "val": 3
}

to_entries

[!NOTE] references:

# original
$ echo '{ "name" : "marslo" }' | jq -r
{
  "name": "marslo"
}

# `to_entries[]`
$ echo '{ "name" : "marslo" }' | jq -r 'to_entries[]'
{
  "key": "name",
  "value": "marslo"
}

# or `to_entries`
$ echo '{"a": 1, "b": 2}' | jq -r to_entries
[
  {
    "key": "a",
    "value": 1
  },
  {
    "key": "b",
    "value": 2
  }
]

  • example to get key and value

    $ echo '{ "some": "thing", "json": "like" }' \
       | jq -r 'to_entries[] | "\(.key)\t\(.value)"'
some    thing
json    like

    • or output format to @csv

      $ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | @csv'
"thing","like"

    • or output format to @tsv

      $ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | @tsv'
thing like

  • to_entries and select

    # orignal
$ echo '{ "name/" : "marslo", "age/" : "18", "citizenship" : "china" }' | jq -r
{
  "name/": "marslo",                 # wants value if key ends with '/'
  "age/": "18",                      # wants value if key ends with '/'
  "citizenship": "china"
}

# select
$ echo '{ "name/" : "marslo", "age/" : "18", "citizenship" : "china" }' |
       jq -r 'to_entries[] | select(.key|endswith("/")) '
{
  "key": "name/",
  "value": "marslo"
}
{
  "key": "age/",
  "value": "18"
}

# get `.value` after selected
$ echo '{ "name/" : "marslo", "age/" : "18", "citizenship" : "china" }' |
       jq -r 'to_entries[] | select(.key|endswith("/")) | .value'
marslo
18

try to_entries

$ cat sample.json
[
  {
    "name": "x",
    "hobby": [                     // < has hobby
      "baseball",
      "baseketball"
     ]
  },
  {
    "name": "y"                    // < no hobby
  }
]

# to_entries[] directly will cause issue
$ cat sample.json |
      jq '.[] | .name as $n | .hobby | to_entries[] | [$n, .value]'
[
  "x",
  "baseball"
]
[
  "x",
  "baseketball"
]
jq: error (at <stdin>:12): null (null) has no keys

# try to_entries[]
#                                      try
#                                       v
$ cat sample.json |
      jq '.[] | .name as $n | .hobby | try to_entries[] | [$n, .value]'
[
  "x",
  "baseball"
]
[
  "x",
  "baseketball"
]

from_entries

$ echo '[{"key":"a", "value":1}, {"key":"b", "value":2}]' |
       jq -r from_entries
{
  "a": 1,
  "b": 2
}

with_entries

[!NOTE|label:references:]

$ echo '{"a": 1, "b": 2}' |
       jq 'with_entries(.key |= "KEY_" + .)'
{
  "KEY_a": 1,
  "KEY_b": 2
}

  • to get particular branch permissions from Gerrit

    # with_entries
$ curl -fsSL https://gerrit.domain.com/a/projects/$(printf %s "path/to/sandbox" | jq -sRr @uri)/access |
       tail -n+2 |
       jq -r '.local | with_entries( if(.key | test("^.+sandbox/marslo.+$")) then ( {key: .key, value: .value } ) else empty end )'

# try to_entries[]
$ curl -fsSL https://gerrit.domain.com/a/projects/$(printf %s "path/to/sandbox" | jq -sRr @uri)/access |
       tail -n+2 |
       jq -r '.local | try to_entries[] | select(.key | test("^.+sandbox/marslo.+$")) | .value'

as

[!NOTE|label:references]

$ echo '{ "name/" : "marslo", "age/" : "18", "citizenship" : "china" }' |
       jq -r '. as $o | keys_unsorted[] | select(endswith("/")) | $o[.]'
marslo
18

tricky

space in the key

$ echo '{ "k1 name": "v1", "k2 name": "v2", "k3": "v3", "k4": "v4" }' |
       jq -r '."k1 name"'
v1

get urlencode

$ printf %s 'input text' | jq -sRr @uri
input%20text

$ printf %s 'input=(text)' | jq -sRr @uri
input%3D%28text%29

$ printf %s '(=)&' | jq -sRr @uri
%28%3D%29%26

string to json

$ echo [
  "foo1:         xxxxxx",
  "foo2:    xxxxxx",
  "foo3:     xxxxxx",
  "foo4:         xxxxxx",
  "foo5:   xxxxxx",
  "foo6:       xxxxxx"
] | jq -r 'map(capture("^(?<key>.*):\\s*(?<value>.*)$")) | from_entries'
{
  "foo1": "xxxxxx",
  "foo2": "xxxxxx",
  "foo3": "xxxxxx",
  "foo4": "xxxxxx",
  "foo5": "xxxxxx",
  "foo6": "xxxxxx"
}

map

[!NOTE|label:references:]

  • semantics of map on a sequence of objects in jq

    • map( ... ) ≡ [ .[] | ... ], which map( . ) ≡ [ .[] | . ] ≡ [ .[] ]

    • for array: map( . ) ≡ [ .[0], .[1], ... ] ≡ .

    • for object: map( . ) ≡ [ .["key1"], .["key2"], ... ]

PrevioushtmlNextregex

Last updated