sqlbulkcopy

Please bear with me as this is my first post. I'm trying to run the COPY command in PostgreSQL-9.2 to add a tab delimited table from a .txt file to a PostgreSQL database such as: COPY raw_data FROM '/home/Projects/TestData/raw_data.txt' WITH (DELIMITER ' '); I've already created an empty table called "raw_data" in the database using the SQL command: CREATE TABLE raw_data (); I keep getting the following error message when trying to run the COPY command: ERROR: extra data after last expected column CONTEXT: COPY raw_data, line 1: " 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

I am trying to read a CSV file with CsvHelper, load each record into a DataTable, and then use SqlBulkCopy to insert the data into a database table. With the current code, I get an exception when adding a row to the DataTable. The exception is: "Unable to cast object of type 'MvcStockAnalysis.Models.StockPrice' to type 'System.IConvertible'.Couldn't store in Date Column. Expected type is DateTime." The example CSV file is from yahoo finance. For example: http://ichart.yahoo.com/table.csv?s=MMM&a=0&b=1&c=2010&d=0&e=17&f=2014&g=d&ignore=.csv The CSV file contains the following header: Date Open

I want to insert many rows (constructed from Entity Framework objects) to SQL Server. The problem is, some of string properties have length exceeded length of column in database, which causes an exception, and then all of rows will unable to insert into database. So I wonder that if there is a way to tell SqlBulkCopy to automatically truncate any over-length rows? Of course, I can check and substring each property if it exceeds the limited length, before insert it in to a DataTable, but it would slow down the whole program. Always use a staging/load table for bulk actions. Then you can process

I dont know why I am getting the above exception, please someone look at it .... DataTable DataTable_Time = new DataTable("Star_Schema__Dimension_Time"); DataColumn Sowing_Day = new DataColumn(); Sowing_Day.ColumnName = "Sowing_Day"; DataColumn Sowing_Month= new DataColumn(); Sowing_Month.ColumnName = "Sowing_Month"; DataColumn Sowing_Year = new DataColumn(); Sowing_Year.ColumnName = "Sowing_Year"; DataColumn Visit_Day= new DataColumn(); Visit_Day.ColumnName = "Visit_Day"; DataColumn Visit_Month = new DataColumn(); Visit_Month.ColumnName = "Visit_Month"; DataColumn Visit_Year = new DataColumn(

I am working to increase the performance of bulk loads; 100's of millions of records + daily. I moved this over to use the IDatareader interface in lieu of the data tables and did get a noticeable performance boost (500,000 more records a minute). The current setup is: A custom cached reader to parse the delimited files. Wrapping the stream reader in a buffered stream. A custom object reader class that enumerates over the objects and implements the IDatareader interface. Then SqlBulkCopy writes to server The bulk of the performance bottle neck is directly in SqlBulkCopy.WriteToServer . If I

In my previous question on this portal, I had asked about some insight about syncing data between SQL Server and key-value based data repositories. In lieu of the same problem (one way real-time synchronization from SQL to HBase or any other database), I need to take care of some performance and latency considerations and did not find a very foolproof way of doing it. We have multiple SQL 2008 data shards where data is updated from various sources and processed by many processes at the same time (and UI reads from the same shards). The goal is to get all updates in selected tables at any point

Pretty straight-forward import using SqlBulkCopy: string excelConnectionString = @"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filePath + ";Extended Properties=\"Excel 12.0 Xml;HDR=YES;IMEX=1;\""; using (OleDbConnection excelConnection = new OleDbConnection(excelConnectionString)) { excelConnection.Open(); OleDbCommand cmd = new OleDbCommand("Select " + fileID.ToString() + " as [FileID], * from [Sheet1$] where [Text] IS NOT NULL", excelConnection); OleDbDataReader dReader = cmd.ExecuteReader(); using (SqlBulkCopy sqlBulk = new SqlBulkCopy(ConfigurationManager.ConnectionStrings[

How to copy data from one DB into another DB with the same table structure and keep the key identities? I use Sql Server 2012 "Denali" and I want to copy some data from a Sql Server 2008 DB. The tables I have are exactly the same but I want the data from the old DB into the new "Denali" DB. The databases are on different servers. So I want something like USE newDB; GO SELECT * INTO newTable FROM OldDb.oldTable WITH (KEEPIDENTITY); GO Anyone have a suggestion to make this workable ? Configure a linked server and reference it in your query. You may need to use IDENTITY_INSERT as well. The SSIS