Adding 5834580 of node to Spatial Layer

不羁的心 提交于 2019-12-24 07:24:08

问题


I am trying to create an R-TREE of 5834580 of nodes.
I found in this question a similair problem and i tried it's solution, so this is my code :

call apoc.periodic.commit("MATCH (pl:pickup_location) WITH collect(pl) AS pickup CALL spatial.addNodes('nyc',pickup) YIELD count RETURN count",{limit:1000})

however, since yesterday the computer didn't finish loading the result.

today, i tried the second answer with iterate :

CALL apoc.periodic.iterate(
"MATCH (pl:pickup_location) RETURN pl",
"CALL spatial.addNode('nyc', pl) YIELD node RETURN node",
{batchSize:10000, parallel:false, listIterate:true})

and i get this error :

Neo.ClientError.Procedure.ProcedureCallFailed: Failed to invoke procedure `apoc.periodic.iterate`: Caused by: java.lang.OutOfMemoryError: Java heap space

what is wrong ? what should i do ?


回答1:


Your problem with the apoc.periodic.commit is that your query will always return the same nodes with your MATCH (pl:pickup_location). There is no condition to find only nodes that are not in the spatial layout.

I don't remember the model of the spatial plugin, but from what I remember, on your pickup_location nodes, you should have a specific relation to the R-Tree.

So you should transform your auery to something like that :

CALL apoc.periodic.commit("
  MATCH (pl:pickup_location)
  WHERE NOT (p1)-[:LINKS->(:spatialNode) // Change this according to the spatial model
  WITH p1 AS node LIMIT $limit
  WITH collect(node) AS pickup 
    CALL spatial.addNodes('nyc',pickup) YIELD count 
    RETURN count",
  {limit:1000}
)

For the problem on the apoc.periodic.iterate is just a memory problem, you don't have enought RAM to execute transactions.

You have two solutions :

  • give more RAM to Neo4j by increasing the heap size of Neo4j (see the neo4j.conf file)
  • decrease the size of the batch, 10000 is a little big, change it to 1000



回答2:


As I don't have a rep of 50 I can't comment but logisima is spot on, just to add to their answer... the "where not" clause should be:

where not (p1)-[:RTREE_REFERENCE]-() 

RTREE_REFERENCE being the relationship created when adding a node to the spatial layer.



来源:https://stackoverflow.com/questions/50988626/adding-5834580-of-node-to-spatial-layer

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!