json

json online player:
Online JSON Formatter & Validator
jq online player:
https://jqplay.org
reference:
jq manual (development version)
Parsing JSON with jq
olih/jq-cheetsheet.md
Use jq to filter objects list with regex
jq: Cannot iterate over number / string and number cannot be added
Guide to Linux jq Command for JSON Processing
Reshaping JSON with jq
* jq cheat sheet
Replacing a missing or null property
Replacing substrings in a string

basic

syntax for jq

SYNTAX

DESCRIPTION

,

Filters separated by a comma will produce multiple independent outputs

?

Will ignores error if the type is unexpected

[]

Array construction

{}

Object construction

+

Concatenate or Add

-

Difference of sets or Substract

length

Size of selected element

⎮

Pipes are used to chain commands in a similar fashion than bash

dealing with json objects

DESCRIPTION

COMMAND

Display all keys

jq 'keys'

Adds + 1 to all items

jq 'map_values(.+1)'

Delete a key

jq 'del(.foo)'

Convert an object to array

to_entries ⎮ map([.key, .value])

slicing and filtering

[!NOTE|label:references:]
select by type: with type been arrays, objects, iterables, booleans, numbers, normals, finites, strings, nulls, values, scalars

DESCRIPTION

COMMAND

All

jq .[]

First

jq '.[0]'

Range

jq '.[2:4]'

First 3

jq '.[:3]'

Last 2

jq '.[-2:]'

Before Last

jq '.[-2]'

split

jq '.[] ⎮ split("/")[1]'

Select array of int by value

jq 'map(select(. >= 2))'

Select array of objects by value

jq '.[] ⎮ select(.id == "second")'

Select by type

jq '.[] ⎮ numbers'

mapping and transforming

DESCRIPTION

COMMAND

Add + 1 to all items

jq 'map(.+1)'

Delete 2 items

jq 'del(.[1, 2])'

Concatenate arrays

jq 'add'

Flatten an array

jq 'flatten'

Create a range of numbers

jq '[range(2;4)]'

Display the type of each item

jq 'map(type)'

Sort an array of basic type

jq 'sort'

Sort an array of objects

jq 'sort_by(.foo)'

Group by a key - opposite to flatten

jq 'group_by(.foo)'

Minimun value of an array

jq 'min' See also min, max, min_by(path_exp), max_by(path_exp)

Remove duplicates

jq 'unique' or jq 'unique_by(.foo)' or jq 'unique_by(length)'

Reverse an array

jq 'reverse'

join

[!TIP] references:
to output multiple values on a single line
join multiple values

$ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | join(":")'
thing:like

$ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | "\(.[0]) \(.[1])"'
thing like

$ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | "\(.[0]): \(.[1])"'
thing: like

or with reduce

$ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | reduce .[1:][] as $i ("\(.[0])"; . + ",\($i)")'
thing,like

$ echo '{ "k1": "v1", "k2": "v2", "k3": "v3", "k4": "v4" }' |
       jq -r '[.k1, .k2, .k3] | reduce .[1:][] as $i ("\(.[0])"; . + ",\($i)")'
v1,v2,v3

or with .first and .last

$ echo '{ "users": [ { "first": "Stevie", "last": "Wonder" }, { "first": "Michael", "last": "Jackson" } ] }' |
       jq -r '.users[] | .first + " " + .last'
Stevie Wonder
Michael Jackson

$ echo '{ "users": [ { "first": "Stevie", "last": "Wonder" }, { "first": "Michael", "last": "Jackson" } ] }' |
       jq -r '.users[] | .first + " " + (.last|tostring)'

$ echo '{ "users": [ { "first": "Stevie", "last": "Wonder", "number": 1 }, { "first": "Michael", "last": "Jackson", "number": 2 } ] }' |
       jq -r '.users[] | .first + " " + (.number|tostring)'
Stevie 1
Michael 2

join with getOrDefault

[!TIP|label:references:]

orignal wihtout getOrDefault

$ echo '[
        {"id": "1", "version": "v1"},
        {"id": "2", "version": "v2"},
        {"id": "3", "version": null}
        ]
       ' |
  jq -r '.[] | .id + " -> " + .version'
1 -> v1
2 -> v2
3 ->

using //

[!TIP|label:references:]
Get or default function in JQ?

$ echo '[
          {"id": "1", "version": "v1"},
          {"id": "2", "version": "v2"},
          {"id": "3", "version": null}
        ]' |
  jq -r '.[] | .id + " -> " + ( .version // "UNKNOWN" )'
1 -> v1
2 -> v2
3 -> UNKNOWN

using if then else end

[!TIP|label:references:]
Filter empty and/or null values with jq
Get or default function in JQ?

$ echo '[
          { "id": "1", "version": "v1" } ,
          { "id": "2", "version": "v2" } ,
          { "id": "3", "version": null } ,
          { "id": "4", "version": ""   }
        ]' |
  jq -r '.[] |
         .id + " -> " +
         if (.version != null and .version != "") then .version
         else "UNKNOWN"
         end'

# or
$ echo '[
          { "id": "1", "version": "v1" } ,
          { "id": "2", "version": "v2" } ,
          { "id": "3", "version": null } ,
          { "id": "4", "version": ""   }
        ]' |
  jq -r '.[] |
         .id + " -> " +
         if .version == null or .version == "" then "UNKNOWN"
         else .version
         end'
1 -> v1
2 -> v2
3 -> UNKNOWN
4 -> UNKNOWN

# or using `empty`
#                           `empty` will not output anything for id==3 and id==4
#                                                     v
$ echo '[
          { "id": "1", "version": "v1" } ,
          { "id": "2", "version": "v2" } ,
          { "id": "3", "version": null } ,
          { "id": "4", "version": ""   }
        ]
       ' |
  jq -r '.[] |
         .id + " -> " +
         if .version == null or .version == "" then empty
         else .version
         end
        '
1 -> v1
2 -> v2
// `empty` will not output anything for id==3 and id==4

# or including non-exists key
$ echo '[
          { "id": "1", "version": "v1" } ,
          { "id": "2", "version": "v2" } ,
          { "id": "3", "version": null } ,
          { "id": "4", "version": ""   } ,
          { "id": "5"                  }
        ]
       ' |
  jq -r '.[] |
         .id + " -> " +
         if ( has("version") and .version != null and .version != "" ) then
           .version
         else "UNKNOWN"
         end
        '
1 -> v1
2 -> v2
3 -> UNKNOWN
4 -> UNKNOWN
5 -> UNKNOWN

split

[!TIP] references:
remove a substring from a string
split(str)
Splits an input string on the separator argument.
example
jq 'split(", ")'
Input "a, b,c,d, e, "
Output  ["a","b,c,d","e",""]

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri'
/1
/2
/3

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri | split("/")[1]'
1
2
3

try online

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq '.[].uri | split("/")[]'
""
"1"
""
"2"
""
"3"

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri | split("/")'
[
  "",
  "1"
]
[
  "",
  "2"
]
[
  "",
  "3"
]

replacing

[!TIP|label:references:]
Replacing substrings in a string
sub(regex; tostring), sub(regex; string; flags)
Emit the string obtained by replacing the first match of regex in the input string with tostring, after interpolation. tostring should be a jq string, and may contain references to named captures. The named captures are, in effect, presented as a JSON object (as constructed by capture) to tostring, so a reference to a captured variable named "x" would take the form: "(.x)".

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri'
/1
/2
/3

$ echo '[{"uri" : "/1" }, {"uri" : "/2"}, {"uri" : "/3"}]' |
        jq -r '.[].uri | sub("/"; "")'
1
2
3

builtin operators

[!NOTE|label:references:]
jq manual - Builtin operators and functions

debug

without debug

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "SECOND", "val": 3}]'''  |
       jq '.[] | select( .val == (2, 3) )'

with debug:

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "SECOND", "val": 3}]'''  |
       jq '.[] | select( .val == (2, 3) | debug )'
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",true]
{
  "id": "second",
  "val": 2
}
["DEBUG:",false]
["DEBUG:",false]
["DEBUG:",true]
{
  "id": "SECOND",
  "val": 3
}

select

[!NOTE|label:refrences:]
imarslo: example on jenkins api analysis
imarslo: example on gerrit api analysis
jq tips : remove emtpy line
Remove all null values
Filter empty and/or null values with jq
to_entries[]
| select(.value | . == null or . == "")
| if .value == "" then .value |= "\"\(.)\"" else . end
| "\(.key): \(.value)"

$ echo '[1,5,3,0,7]' |
       jq 'map(select(. >= 2))'
[
  5,
  3,
  7
]

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2}]''' |
       jq '.[] | select(.id == "second")'
{
  "id": "second",
  "val": 2
}

select with any

[!NOTE|label:references:]
any(.attributes; .enabled == true) == any(.attributes; .enabled) == .attributes.enabled == true == .attributes.enabled
try online

// sample.json
[
  {
    "attributes": {
      "created": "2021-10-18T12:02:39+00:00",
      "enabled": true,
      "expires": null,
      "notBefore": null
    },
    "contentType": null,
    "id": "https://kjkljk./secrets/-/1",
    "managed": null,
    "name": "pw",
    "tags": {}
  },
  {
    "attributes": {
      "created": "2021-10-18T12:06:16+00:00",
      "enabled": false,
      "expires": null,
      "notBefore": null
    },
    "contentType": "",
    "id": "https://kjklj./secrets/-/2",
    "managed": null,
    "name": "pw",
    "tags": {}
  }
]

# get [{.id}] format
$ cat sample.json |
      jq -r 'map(select(any(.attributes; .enabled)) | {id})'
[
  {
    "id": "https://kjkljk./secrets/-/1"
  }
]

# get [.id] format
$ cat sample.json |
      jq -r 'map(select(any(.attributes; .enabled)) | .id)'
[
  "https://kjkljk./secrets/-/1"
]

# get .id format ( without `map()` )
$ cat sample.json |
      jq -r '.[] | select(any(.attributes; .enabled)) | .id'
https://kjkljk./secrets/-/1

# re-format
$ cat sample.json |
      jq -r '{"ids": .[] | select(any(.attributes; .enabled)) | .id}'
{
  "ids": "https://kjkljk./secrets/-/1"
}

contains

[!NOTE|label:reference:]
imarslo: example on artifactory api analysis
imarslo: example on list Error pods in kuberetnes

$ echo '[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "second-one", "val": 3}]' |
       jq '.[] | select( .id | contains("second") )'
{
  "id": "second",
  "val": 2
}
{
  "id": "second-one",
  "val": 3
}

by using test

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "second.one", "val": 3}]'''  |
       jq '.[] | select( .id | test("sec*") )'
{
  "id": "second",
  "val": 2
}
{
  "id": "second.one",
  "val": 3
}

`inside`

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "second-one", "val": 3}]'''  |
       jq '.[] | select( [.val] | inside([2,3]) )'
{
  "id": "second",
  "val": 2
}
{
  "id": "second-one",
  "val": 3
}

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "SECOND", "val": 3}]'''  |
       jq '.[] | select( .val == (2, 3) )'
{
  "id": "second",
  "val": 2
}
{
  "id": "SECOND",
  "val": 3
}

toUpperCase : `ascii_upcase`

$ echo '''[{"id": "first", "val": 1}, {"id": "second", "val": 2},  {"id": "SECOND", "val": 3}]'''  |
       jq '.[] | select(.id | ascii_upcase == "SECOND")'
{
  "id": "second",
  "val": 2
}
{
  "id": "SECOND",
  "val": 3
}

to_entries

[!NOTE] references:
jq: filter input based on if key ends with specified string

# original
$ echo '{ "name" : "marslo" }' | jq -r
{
  "name": "marslo"
}

# `to_entries[]`
$ echo '{ "name" : "marslo" }' | jq -r 'to_entries[]'
{
  "key": "name",
  "value": "marslo"
}

# or `to_entries`
$ echo '{"a": 1, "b": 2}' | jq -r to_entries
[
  {
    "key": "a",
    "value": 1
  },
  {
    "key": "b",
    "value": 2
  }
]

example to get key and value

$ echo '{ "some": "thing", "json": "like" }' |
       jq -r 'to_entries[] | "\(.key)\t\(.value)"'
some    thing
json    like

or output format to @csv

$ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | @csv'
"thing","like"

or output format to @tsv

$ echo '{ "some": "thing", "json": "like" }' |
       jq -r '[.some, .json] | @tsv'
thing like

to_entries and select

# orignal
$ echo '{ "name/" : "marslo", "age/" : "18", "citizenship" : "china" }' | jq -r
{
  "name/": "marslo",                 # wants value if key ends with '/'
  "age/": "18",                      # wants value if key ends with '/'
  "citizenship": "china"
}

# select
$ echo '{ "name/" : "marslo", "age/" : "18", "citizenship" : "china" }' |
       jq -r 'to_entries[] | select(.key|endswith("/")) '
{
  "key": "name/",
  "value": "marslo"
}
{
  "key": "age/",
  "value": "18"
}

# get `.value` after selected
$ echo '{ "name/" : "marslo", "age/" : "18", "citizenship" : "china" }' |
       jq -r 'to_entries[] | select(.key|endswith("/")) | .value'
marslo
18

try to_entries

// sample.json
[
  {
    "name": "x",
    "hobby": [                     // < has hobby
      "baseball",
      "baseketball"
     ]
  },
  {
    "name": "y"                    // < no hobby
  }
]

# to_entries[] directly will cause issue
$ cat sample.json |
      jq '.[] | .name as $n | .hobby | to_entries[] | [$n, .value]'
[
  "x",
  "baseball"
]
[
  "x",
  "baseketball"
]
jq: error (at <stdin>:12): null (null) has no keys

# try to_entries[]
#                                      try
#                                       v
$ cat sample.json |
      jq '.[] | .name as $n | .hobby | try to_entries[] | [$n, .value]'
[
  "x",
  "baseball"
]
[
  "x",
  "baseketball"
]

from_entries

$ echo '[{"key":"a", "value":1}, {"key":"b", "value":2}]' |
       jq -r from_entries
{
  "a": 1,
  "b": 2
}

with_entries

[!NOTE|label:references:]
JQ: Filtering for keys

$ echo '{"a": 1, "b": 2}' |
       jq 'with_entries(.key |= "KEY_" + .)'
{
  "KEY_a": 1,
  "KEY_b": 2
}

to get particular branch permissions from Gerrit

# with_entries
$ curl -fsSL https://gerrit.domain.com/a/projects/$(printf %s "path/to/sandbox" | jq -sRr @uri)/access |
       tail -n+2 |
       jq -r '.local | with_entries( if(.key | test("^.+sandbox/marslo.+$")) then ( {key: .key, value: .value } ) else empty end )'

# try to_entries[]
$ curl -fsSL https://gerrit.domain.com/a/projects/$(printf %s "path/to/sandbox" | jq -sRr @uri)/access |
       tail -n+2 |
       jq -r '.local | try to_entries[] | select(.key | test("^.+sandbox/marslo.+$")) | .value'

as

[!NOTE|label:references]
jq: filter input based on if key ends with specified string

$ echo '{ "name/" : "marslo", "age/" : "18", "citizenship" : "china" }' |
       jq -r '. as $o | keys_unsorted[] | select(endswith("/")) | $o[.]'
marslo
18

tricky

space in the key

$ echo '{ "k1 name": "v1", "k2 name": "v2", "k3": "v3", "k4": "v4" }' |
       jq -r '."k1 name"'
v1

get urlencode

[!TIP|label:references:]
Percent-encoding

$ printf %s 'input text' | jq -sRr @uri
input%20text

$ printf %s 'input=(text)' | jq -sRr @uri
input%3D%28text%29

$ printf %s '(=)&' | jq -sRr @uri
%28%3D%29%26

string to json

$ echo [
  "foo1:         xxxxxx",
  "foo2:    xxxxxx",
  "foo3:     xxxxxx",
  "foo4:         xxxxxx",
  "foo5:   xxxxxx",
  "foo6:       xxxxxx"
] | jq -r 'map(capture("^(?<key>.*):\\s*(?<value>.*)$")) | from_entries'
{
  "foo1": "xxxxxx",
  "foo2": "xxxxxx",
  "foo3": "xxxxxx",
  "foo4": "xxxxxx",
  "foo5": "xxxxxx",
  "foo6": "xxxxxx"
}

map

[!NOTE|label:references:]
semantics of map on a sequence of objects in jq
map( ... ) ≡ [ .[] | ... ], which map( . ) ≡ [ .[] | . ] ≡ [ .[] ]
for array: map( . ) ≡ [ .[0], .[1], ... ] ≡ .
for object: map( . ) ≡ [ .["key1"], .["key2"], ... ]

Previoushtml Nextregex

Last updated 11 months ago