change json file by bash script

Question

I need your help to solve the following problem: I have a JSON file that looks like this:

{
  \"key1\": \"value1\",
  \"key2\": \"value2\",
  \"key3\": \"val         


        

Answer 1

Your best bet is to use a JSON CLI such as jq:

• On Debian-based systems such as Ubuntu, you can install it via sudo apt-get install jq
• On macOS, with Homebrew (http://brew.sh/) installed, use brew install jq

Examples, based on the following input string - output is to stdout:

jsonStr='{ "key1": "value1", "key2": "value2", "key3": "value3" }'

Remove "key3":

jq 'del(.key3)' <<<"$jsonStr"

Add property "key4" with value "value4":

jq '. + { "key4": "value4" }' <<<"$jsonStr"

Change the value of existing property "key1" to "new-value1":

jq '.key1 = "new-value1"' <<<"$jsonStr"

A more robust alternative ^{thanks, Lars Kiesow} :
If you pass the new value with --arg, jq takes care of properly escaping the value:

jq '.key1 = $newVal' --arg newVal '3 " of rain' <<<"$jsonStr"

---

If you want to update a JSON file in place (conceptually speaking), using the example of deleting "key3":

# Create test file.
echo '{ "key1": "value1", "key2": "value2", "key3": "value3" }' > test.json

# Remove "key3" and write results back to test.json (recreate it with result).
jq -c 'del(.key3)' test.json > tmp.$$.json && mv tmp.$$.json test.json

You cannot replace the input file directly, so the result is written to a temporary file that replaces the input file on success.

Note the -c option, which produces compact rather than pretty-printed JSON.

For all options and commands, see the manual at http://stedolan.github.io/jq/manual/.

Answer 2

how can I add and remove a new key (i.e "key4": "value4") by bash script?

Using a dedicated JSON tool, like Xidel, would be a better idea than to use pure Bash functions.

Add a new attribute-value pair:

Dot notation:

echo '{"a":1,"b":2,"c":3}' | xidel -s - -e '($json).d:=4'
{
  "d": 4,
  "a": 1,
  "b": 2,
  "c": 3
}

JSONiq:

echo '{"a":1,"b":2,"c":3}' | xidel -s - -e '{|$json,{"d":4}|}'
{
  "a": 1,
  "b": 2,
  "c": 3,
  "d": 4
}

XQuery 3.1 (requires Xidel 0.9.9-beta):

echo '{"a":1,"b":2,"c":3}' | xidel -s - -e 'map:put($json,"d",4)'
{
  "d": 4,
  "a": 1,
  "b": 2,
  "c": 3
}

echo '{"a":1,"b":2,"c":3}' | xidel -s - -e 'map:merge(($json,{"d":4}))'
{
  "a": 1,
  "b": 2,
  "c": 3,
  "d": 4
}

Remove the attribute-value pair "d":4:

JSONiq:

echo '{"a":1,"b":2,"c":3,"d":4}' | xidel -s - --xmlns:jnlib="http://jsoniq.org/function-library" -e 'jnlib:remove-keys($json,"d")'
{
  "a": 1,
  "b": 2,
  "c": 3
}

XQuery 3.1 (requires Xidel 0.9.9-beta):

echo '{"a":1,"b":2,"c":3,"d":4}' | xidel -s - -e 'map:remove($json,"d")'
{
  "a": 1,
  "b": 2,
  "c": 3
}

Answer 3

to change a file in place, use the sponge command, like

cat file.json | jq '.path.to.key = $v' --arg v 'new value' | sponge file.json

Answer 4

Building off Lenny's answer, we can use node's -p option, which evaluates the given script and writes the output to stdout.

Using the spread operator for easy modification gives:

node -p "JSON.stringify({...require('./data.json'), key4: 'value4'}, null, 2)" > data.json

Answer 5

Not the answer for everyone, but if you already happen to have NodeJs installed in your system, you can use it to easily manipulate JSON.

eg:

#!/usr/bin/env bash
jsonFile=$1;

node > out_${jsonFile} <<EOF
//Read data
var data = require('./${jsonFile}');

//Manipulate data
delete data.key3
data.key4 = 'new value!';

//Output data
console.log(JSON.stringify(data));

EOF

Heck, if you only need to do JSON manipulation and you have node (ie: You don't really need any other bash functionality) you could directly write a script using node as the interpreter:

#! /usr/bin/env node
var data = require('./'+ process.argv[2]);
/*manipulate*/
console.log(JSON.stringify(data));