How could I curl for a request with headers in shell scripting

Question

I tried to execute a CURL statement as follows, for which I am getting required response:

curl -s -POST --header 'Content-Type: application/json' 'http://www.dummy.com/projectname/page_relevance' -d '{"query": "q_string", "results": [{"abstract": "abs_string", "title": "title_string"}, "mode": "value", "cache": true, "source": "value"}'

But when i tried to pass variable values to parameter "query", the curl statement mentioned below not works and observed some error statement in response:

curl -s -POST --header 'Content-Type: application/json' 'http://www.dummy.com/projectname/page_relevance' -d '{"query": "$query_string", "results": [{"abstract": "abs_string", "title": "title_string"}, "mode": "value", "cache": true, "source": "value"}'

ERROR Statement:

Not Found [CFN #0005]

It works!

This is the default web page for this server.

The web server software is running but no content has been added, yet.

But am sure that curl request which i constructed second with variable resembles the same curl request which i executed at first. This is tested using echo which replaces $query_string with correct value.

I also tried in another approach, in which i have not used any variables for single parameter, instead i tried as below:

a='{"query": "query_value", "results": [{"abstract": "abs_string", "title": "title_string"}, "mode": "value", "cache": true, "source": "value"}'
curl -s -POST --header 'Content-Type: application/json' 'http://www.dummy.com/projectname/page_relevance' -d $a

I also tried to substitute value of a using ${a}, "$a", '$a'

still same error is been observed.

Answer 1

Single quotes ' (you're using in -d argument) preserve the literal value of each character, including the $ (see this SO answer), and that's why your variable query_string is not being expanded.

Try this:

~$ query_string="my query"

~$ echo '$query_string'
$query_string

~$ echo "$query_string"
my query

So, you need to use double quotes " if you wish your variables to expand to its values.

However, in order to nest double quotes (inside other double quotes), as in you JSON data, you must either:

1. escape the inner quotes, like this:

   ~$ echo "{\"query\": \"$query_string\"}"
   {"query": "my query"}
   

   but that gets very ugly, very soon; or

2. concatenate strings under alternating single and double quotes, like this:

   ~$ echo '{"query": "'"$query_string"'"}"'
   {"query": "my query"}"
   

   which may be more readable for shorter strings; or

3. use a here-document:

   ~$ read query <<-END
   {"query": "$query_string"}
   END
   
   ~$ echo "$query"
   {"query": "my query"}
   

   Here-documents are particularly convenient for longer documents in which you wish for parameter/variable expansion, command substitution, arithmetic expansion, etc.

In summary, after defining your JSON query with one of the above ways (perhaps via a here-document), you can write your curl command like this:

curl -s -X POST -H 'Content-Type: application/json' 'http://www.dummy.com/projectname/page_relevance' -d "$query"