I am struggling with importing data into Mongodb from a Json file.
I can do the same in command line by using mongoimport command.
I just faced this issue today and solved it in another different way while none here satisfied me, so enjoy my extra contribution. Performances are sufficient to export 30k documents and import them in my Springboot app for integration test cases (takes a few seconds).
First, the way your export your data in the first place matters. I wanted a file where each line contains 1 document that I can parse in my java app.
mongo db --eval 'db.data.find({}).limit(30000).forEach(function(f){print(tojson(f, "", true))})' --quiet > dataset.json
Then I get the file from my resources folder, parse it, extract lines, and process them with mongoTemplate. Could use a buffer.
@Autowired
private MongoTemplate mongoTemplate;
public void createDataSet(){
mongoTemplate.dropCollection("data");
try {
InputStream inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(DATASET_JSON);
List documents = new ArrayList<>();
String line;
InputStreamReader isr = new InputStreamReader(inputStream, Charset.forName("UTF-8"));
BufferedReader br = new BufferedReader(isr);
while ((line = br.readLine()) != null) {
documents.add(Document.parse(line));
}
mongoTemplate.insert(documents,"data");
} catch (Exception e) {
throw new RuntimeException(e);
}
}