Insert multiple documents in Elasticsearch - bulk doc formatter

Question

TLDR; How can I bulk format my JSON file for ingestion to Elasticsearch?

I am attempting to ingest some NOAA data into Elasticsearch and have been utilizing NOAA Py

Answer 1

The problem is that the alerts JSON dump is all on a single line, so it's not gonna work as it is. You need to iterate on all the alerts (I suspect whatever is in the alerts.features array) and do it all in one shot without going through an intermediary file, like this:

n = noaa.NOAA()
alerts = n.alerts()
f = open('nhc_alerts.json', 'w')
for alert in alerts['features']:
  f.write('%s\n' % json.dumps({'index': {}}));
  f.write('%s\n' % json.dumps(alert, indent=0).replace('\n', ''))
f.write('\n')