What is the fastest way to load an XML file into MySQL using C#?

Question

Question

What is the fastest way to dump a large (> 1GB) XML file into a MySQL database?

Data

The data in question is the StackOverflow Creative

Answer 1

SqlBulkCopy ROCKS. I used it to turn a 30 min function to 4 seconds. However this is applicable only to MS SQL Server.

Might I suggest you look at the constraints on your table you've created? If you drop all keys on the database, constraints etc, the database will do less work on your insertions and less recursive work.

Secondly setup the tables with big initial sizes to prevent your resizes if you are inserting into a blank database.

Finally see if there is a bulk copy style API for MySQL. SQL Server basically formats the data as it would go down to disk and the SQL server links the stream up to the disk and you pump in data. It then performs one consistency check for all the data instead of one per insert, dramatically improving your performance. Good luck ;)

Do you need MySQL? SQL Server makes your life easier if you are using Visual Studio and your database is low performance/size.