neo4j 2.1.1 How to setup logging to analyze “Unknown Error”

问题

I am trying to load lot of data exported from my SQL Server as csv file in to neo4j using the following query:

USING PERIODIC COMMIT 1000
LOAD CSV WITH HEADERS FROM "file:e:/temp/sql_backup/events.csv" AS csvLine
MERGE (dtStart:Date { Name: REPLACE (SUBSTRING(csvLine.VT_Start,0,10),"-","")})
MERGE (dtEnd:Date { Name: REPLACE (SUBSTRING(csvLine.VT_End,0,10),"-","")})
MERGE (ev:Event  { EventId : csvLine.EventID})
ON CREATE SET ev = {
    EventId : csvLine.EventID,
    TagId : csvLine.TagID,  
    EventTypeId : csvLine.EventTypeID, 
    IntervalIdentifier : csvLine.AlarmID,    
    VT_Start : csvLine.VT_Start ,  
    VT_End : csvLine.VT_End,  
    Suppressed : csvLine.Suppressed, 
    system_messagetypename : csvLine.system_messagetypename,                                
    system_inputname : csvLine.system_inputname,
    system_spare1 : csvLine.system_spare1,
    Priority : csvLine.Priority,
    Console : csvLine.Console,
    Operator : csvLine.Operator,
    Message : csvLine.Message,
    Parameter : csvLine.Parameter,
    FromValue : csvLine.FromValue,
    ToValue : csvLine.ToValue,
    UnitOfMeasure : csvLine.UnitOfMeasure,
    Limit : csvLine.Limit,
    Value : csvLine.Value,
    User1 : csvLine.User1,
    BlockName : csvLine.BlockName }

MERGE (al:Alarm{ Name:  csvLine.AlarmID})
MERGE (pr:Priority{ Name:  csvLine.Priority})
MERGE (con:Console{ Name:  csvLine.Console })
MERGE (op:Operator{ Name:  csvLine.Operator })
MERGE (prm:Parameter{ Name:  csvLine.Parameter })
MERGE (uom:UOM{ Name:  csvLine.UnitOfMeasure })
MERGE (user:User{ Name:  csvLine.User1 })
MERGE (blk:Block{ Name:  csvLine.BlockName })
MERGE (tag:Tag{ TagId:  csvLine.TagID })
MERGE (evt:EventType{ EventTypeID:  csvLine.EventTypeID })
CREATE UNIQUE  (con)<-[:FOR_CONSOLE]-(ev)-[:HAS_ALARM]->(al)
CREATE UNIQUE  (op)<-[:FOR_OPERATOR]-(ev)-[:HAS_PRIORITY]->(pr)
CREATE UNIQUE  (user)<-[:FOR_USER]-(ev)-[:HAS_UOM]->(uom)
CREATE UNIQUE  (blk)<-[:HAS_BLOCK]-(ev)-[:HAS_EVENT_TYPE]->(evt)
CREATE UNIQUE  (ev)-[:FOR_TAG]->(tag)
CREATE UNIQUE  (dtStart)<-[:DATE_VT_START]-(ev)-[:FOR_TAG]->(tag)-[:DATE_VT_END]->(dtEnd)

I see that this query runs for a while and i get an error like below:

I looked in to the data directory to check if there was any logs there that explains more details, but I can't find any logs generated there as well.

neo4j documents talks about editing conf/neo4j-server.properties but i don't see either a config folder or logs in my neo4j directory as shown below.

only logs I see in that root folder is nioneo_logical.log, tm_tx_log, active_tx_log

Can someone please explain me how to setup debug logging in neo4j 2.1.1 so that I can see whats causing this error. I suspect it could be that my Laptop is running out of available RAM may be, so want to know if anyone think that neo4j crashes when it runs out available of RAM.

Also I noticed that the some data is created before the error is thrown, but no relationships queries are created, so is there anything wrong in the query itself? or its the error that is causing the issue of not creating the relationships.

EDIT 1:

BTW what happened to the neo4j 2.1.1 download? i don't find it in the neo4j site anymore !

EDIT 2: I downloaded the latest version 2.1.2 and tried to run the query again and landed in the same issue. I think I kind of get the issue, the periodic commit is not useful as it looks to me that query first creates all the event nodes and then starts to run the create associations.

I reduced the size of events to 5000 and i got it to work, but when I increased it to 100000 it crashed again and there were 30000+ events in the database with no associations.

My conclusion is that: Either my query is incorrect, or the way Periodic commits are handled is not correct. This way we will run out of physical RAM when the dataset is large.

Edit 3:

Here is the output from the Shell, where the 1st query runs as it does not have any Create statements.

Same runs from the Browser window though if the file size is not large.

回答1:

The reason that you cannot find any errors in the logs is that long running queries in the Neo4j browser timeout at 60 seconds. This does not mean that the query actually failed, in fact it will run to completion.

For testing long running queries, please use the Neo4j Shell, which does not have a timeout.

http://docs.neo4j.org/chunked/stable/shell.html

来源：https://stackoverflow.com/questions/24134969/neo4j-2-1-1-how-to-setup-logging-to-analyze-unknown-error

标签

neo4j