how to speed up insertion in neo4j from mysql?

左心房为你撑大大i 提交于 2019-12-11 06:51:19

问题


I have a dataset of 60000 items in mysql and i'm trying to insert it into neo4j. The insertion is taking place but it's taking a long time( approx. 10-15 per 3 sec). Is there any way i can speed it up? also is there any way i could give something such as unique key in neo4j so duplication indexes won't get indexed? I'm new to neo4j.

I'm using neo4j 1.8 with PHP Everyman driver.


回答1:


There is a nice presentation from Max De Marzi, about ETL into Neo4j.

See: http://www.slideshare.net/maxdemarzi/etl-into-neo4j

It depends which language you want to use, lots of options from java embedded via jruby and remotely via ruby, php, python.

You would want to batch your requests in appropriately sized transactions (e.g. 10k items per tx).

It is possible to import CSV files directly into a database file using my the batch-importer or via the BATCH REST API of the Neo4j Server.




回答2:


As mentioned above, the preferred option is the batch importer.

If you need to go through the PHP client, here I've put up an example that uses the REST batch api: http://phpfiddle.org/main/code/mu3-sgk

You can test the rows/batch what works best for your system. For my laptop it's 750, for my test-server it's 1250. The json_decode that happens is heavy on the CPU.



来源:https://stackoverflow.com/questions/13686850/how-to-speed-up-insertion-in-neo4j-from-mysql

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!