bulkinsert

First off let me say I am running on SQL Server 2005 so I don't have access to MERGE . I have a table with ~150k rows that I am updating daily from a text file. As rows fall out of the text file I need to delete them from the database and if they change or are new I need to update/insert accordingly. After some testing I've found that performance wise it is exponentially faster to do a full delete and then bulk insert from the text file rather than read through the file line by line doing an update/insert. However I recently came across some posts discussing mimicking the MERGE functionality

In our current project, customers will send collection of a complex/nested messages to our system. Frequency of these messages are approx. 1000-2000 msg/per seconds. These complex objects contains the transaction data (to be added) as well as master data (which will be added if not found). But instead of passing the ids of the master data, customer passes the 'name' column. System checks if master data exist for these names. If found, it uses the ids from database otherwise create this master data first and then use these ids. Once master data ids are resolved, system inserts the transactional

Hi I'm working a project where we need to process several xml files once a day and populate a Database with the information contained in those files. Each file is roughly 1Mb and contains about 1000 records; we usually need to process between 12 and 25 of these files. I've seen some information regarding bulk inserts using NHibernate but our problem is somehow trickier since the xml files contain new records mixed with updated records. In the xml there is a flag that tells us is a specific record is a new one or an update to an existing record, but not what information has changed. The xml

I am trying to insert the data from this link to my SQL server https://www.ian.com/affiliatecenter/include/V2/CityCoordinatesList.zip I created the table CREATE TABLE [dbo].[tblCityCoordinatesList]( [RegionID] [int] NOT NULL, [RegionName] [nvarchar](255) NULL, [Coordinates] [nvarchar](4000) NULL ) ON [PRIMARY] And I am running the following script to do the bulk insert BULK INSERT tblCityCoordinatesList FROM 'C:\data\CityCoordinatesList.txt' WITH ( FIRSTROW = 2, MAXERRORS = 0, FIELDTERMINATOR = '|', ROWTERMINATOR = '\n' ) But the bulk insert fails with following error Cannot obtain the

Please bear with me as I explain the problem, how I tried to solve it, and my question on how to improve it is at the end. I have a 100,000 line csv file from an offline batch job and I needed to insert it into the database as its proper models. Ordinarily, if this is a fairly straight-forward load, this can be trivially loaded by just munging the CSV file to fit a schema; but, I had to do some external processing that requires querying and it's just much more convenient to use SQLAlchemy to generate the data I want. The data I want here is 3 models that represent 3 pre-exiting tables in the

I have a CSV file at the client side, and I want to develop a C# application to bulk insert the data into a table of a database to minimal log output. I am confused about if I use ADO.NET at the client side to call stored procedures in the database server. What kind of code needs to develop at the client side and what kind of code needs to be implemented at the server side in the form of stored procedures? But I did not find any samples from Google. What are some ready to use samples? :-) EDIT: Some more information: I have a lot of data at the client side and I want to import to the database,