Neo4jClient - Merge within ForEach with 1000 very slow (Unique Constraint)

问题

This stems off of this SO question: Map params with Merge or build a Batch operation via the client?

This query:

FOREACH (n in {set} | MERGE (c:Label {Id : n.Id}) SET c = n)

Works well for being able to update or create new nodes based on a unique key (in this case, Id). It is, however, very slow. When I try to process a list of 42000 items, in batches of 1000, it takes about 2 minutes to complete. In terms of hardware, this is running on my laptop (i7, 8GB RAM, Samsung 840 Pro SSD). By the way, this is running on a clean database, so initially, there are no Matches, just creates. (Although I want it to support updating the data, hence the use of Merge.

C# code:

        createUniqueConstraint(label, PK_Field); //Creating a constraint automatically creates an index
        string propKey = label + "s" ;
        string strForEach = string.Format("(n in {{{0}}} |  MERGE (c:{1} {{{2} : n.{2}}}) SET c = n)",
            propKey, label, PK_Field);
        foreach (var entities in list.Batch(1000))
        {
            var query = client
                            .Cypher
                            .ForEach(strForEach)
                            .WithParam(propKey, entities.ToList());

            query.ExecuteWithoutResults();
            Debug.WriteLine("Batch passed for " + label + " ("+entities.Count()+" items)");
        }

UPDATE

If I just manually insert each list of node types using Merge and foreach (with a unique constraint), CPU sits at 32% and it takes a long time per list of 40-50k items. Getting the data for the 50k items takes about 2 seconds.

(I ran this with pulling data without executing the insert query and I could see the data pull took max 2 seconds)

This is the QueryText while running (it's still doing the first batch of 42k)

[Neo4jClient.Cypher.CypherFluentQuery] = 
"FOREACH (n in System.Collections.Generic.List`1[Common.Models.Contact] |
 MERGE (c:Contact {ContactId : n.ContactId}) SET c = n)"

Screenshot:

来源：https://stackoverflow.com/questions/21908230/neo4jclient-merge-within-foreach-with-1000-very-slow-unique-constraint

标签

neo4j

neo4jclient