Fastest way to add new node to end of an xml?

Question

I have a large xml file (approx. 10 MB) in following simple structure:

   .......
   .......

        

Answer 1

How is your XML-File represented in code? Do you use the System.XML-classes? In this case you could use XMLDocument.AppendChild.

Answer 2

First, I would disqualify System.Xml.XmlDocument because it is a DOM which requires parsing and building the entire tree in memory before it can be appended to. This means your 10 MB of text will be more than 10 MB in memory. This means it is "memory intensive" and "time consuming".

Second, I would disqualify System.Xml.XmlReader because it requires parsing the entire file first before you can get to the point of when you can append to it. You would have to copy the XmlReader into an XmlWriter since you can't modify it. This requires duplicating your XML in memory first before you can append to it.

The faster solution to XmlDocument and XmlReader would be string manipulation (which has its own memory issues):

string xml = @"<Errors><error />...<error /></Errors>";
int idx = xml.LastIndexOf("</Errors>");

xml = xml.Substring(0, idx) + "<error>new error</error></Errors>";

Chop off the end tag, add in the new error, and add the end tag back.

I suppose you could go crazy with this and truncate your file by 9 characters and append to it. Wouldn't have to read in the file and would let the OS optimize page loading (only would have to load in the last block or something).

System.IO.FileStream fs = System.IO.File.Open("log.xml", System.IO.FileMode.Open, System.IO.FileAccess.ReadWrite);
fs.Seek(-("</Errors>".Length), System.IO.SeekOrigin.End);
fs.Write("<error>new error</error></Errors>");
fs.Close();

That will hit a problem if your file is empty or contains only "<Errors></Errors>", both of which can easily be handled by checking the length.

Answer 3

Using string-based techniques (like seeking to the end of the file and then moving backwards the length of the closing tag) is vulnerable to unexpected but perfectly legal variations in document structure.

The document could end with any amount of whitespace, to pick the likeliest problem you'll encounter. It could also end with any number of comments or processing instructions. And what happens if the top-level element isn't named Error?

And here's a situation that using string manipulation fails utterly to detect:

<Error xmlns="not_your_namespace">
   ...
</Error>

If you use an XmlReader to process the XML, while it may not be as fast as seeking to EOF, it will also allow you to handle all of these possible exception conditions.

Answer 4

The fastest way would probably be a direct file access.

using (StreamWriter file = File.AppendText("my.log"))
{
    file.BaseStream.Seek(-"</Errors>".Length, SeekOrigin.End);
    file.Write("   <Error>New error message.</Error></Errors>");
}

But you lose all the nice XML features and may easily corrupt the file.