is there any way to import a json file(contains 100 documents) in elasticsearch server.?

Question

Is there any way to import a JSON file (contains 100 documents) in elasticsearch server? I want to import a big json file into es-server..

Answer 1

You can use esbulk, a fast and simple bulk indexer:

$ esbulk -index myindex file.ldj

Here's an asciicast showing it loading Project Gutenberg data into Elasticsearch in about 11s.

Disclaimer: I'm the author.

Answer 2

You should use Bulk API. Note that you will need to add a header line before each json document.

$ cat requests
{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -XPOST localhost:9200/_bulk --data-binary @requests; echo
{"took":7,"items":[{"create":{"_index":"test","_type":"type1","_id":"1","_version":1,"ok":true}}]}

Answer 3

As dadoonet already mentioned, the bulk API is probably the way to go. To transform your file for the bulk protocol, you can use jq.

Assuming the file contains just the documents itself:

$ echo '{"foo":"bar"}{"baz":"qux"}' | 
jq -c '
{ index: { _index: "myindex", _type: "mytype" } },
. '

{"index":{"_index":"myindex","_type":"mytype"}}
{"foo":"bar"}
{"index":{"_index":"myindex","_type":"mytype"}}
{"baz":"qux"}

And if the file contains the documents in a top level list they have to be unwrapped first:

$ echo '[{"foo":"bar"},{"baz":"qux"}]' | 
jq -c '
.[] |
{ index: { _index: "myindex", _type: "mytype" } },
. '

{"index":{"_index":"myindex","_type":"mytype"}}
{"foo":"bar"}
{"index":{"_index":"myindex","_type":"mytype"}}
{"baz":"qux"}

jq's -c flag makes sure that each document is on a line by itself.

If you want to pipe straight to curl, you'll want to use --data-binary @-, and not just -d, otherwise curl will strip the newlines again.