问题
I'm trying to use the SqlBulkCopy
class from the System.Data
assembly (4.6.1) to bulk insert a table with a geospatial data type, using code that looks roughly like this (adapted from https://github.com/MikaelEliasson/EntityFramework.Utilities):
public void InsertItems<T>(IEnumerable<T> items, string schema, string tableName, IList<ColumnMapping> properties, DbConnection storeConnection, int? batchSize)
{
using (var reader = new EFDataReader<T>(items, properties))
{
var con = (SqlConnection)storeConnection;
if (con.State != ConnectionState.Open)
{
con.Open();
}
using (var copy = new SqlBulkCopy(con))
{
copy.BatchSize = batchSize ?? 15000; //default batch size
if (!string.IsNullOrWhiteSpace(schema))
{
copy.DestinationTableName = $"[{schema}].[{tableName}]";
}
else
{
copy.DestinationTableName = "[" + tableName + "]";
}
copy.NotifyAfter = 0;
foreach (var i in Enumerable.Range(0, reader.FieldCount))
{
copy.ColumnMappings.Add(i, properties[i].NameInDatabase);
}
copy.WriteToServer(reader); // <-- throws here
copy.Close();
}
}
}
That works great, until I try to use it on a table with geospatial data. When I do that, I get the following error:
ERROR Swyfft.Console.TaskManager - Error running task SeedRating:
(InvalidOperationException) The given value of type DbGeography from the data source cannot be converted to type udt of the specified target column.;
(ArgumentException) Specified type is not registered on the target server.System.Data.Entity.Spatial.DbGeography, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089.;
at Swyfft.Data.Utilities.SqlQueryProvider.InsertItems[T](IEnumerable`1 items, String schema, String tableName, IList`1 properties, DbConnection storeConnection, Nullable`1 batchSize) in C:\source\swyfft\swyf-website\Swyfft.Data.Utilities\SqlQueryProvider.cs:line 78
at Swyfft.Data.Utilities.EFBatchOperation`2.InsertAll[TEntity](IEnumerable`1 items, DbConnection connection, Nullable`1 batchSize) in C:\source\swyfft\swyf-website\Swyfft.Data.Utilities\EFBatchOperation.cs:line 138
at Swyfft.Data.Rating.RatingContext.BulkInsert[T](IEnumerable`1 entities, Nullable`1 batchSize) in C:\source\swyfft\swyf-website\Swyfft.Data.Rating\RatingContext.cs:line 69
at Swyfft.Seeding.CsvLoaders.CsvLoader.ProcessCsv[T](StreamReader streamReader, String fileName, ISwyfftContext ctx, Func`2 parserFunc) in C:\source\swyfft\swyf-website\Swyfft.Seeding\CsvLoaders\CsvLoader.cs:line 133
at Swyfft.Seeding.CsvLoaders.CsvLoader.InitializeCountyBlockQualities(String stateFilter) in C:\source\swyfft\swyf-website\Swyfft.Seeding\CsvLoaders\InitializeCountyBlockQualities.cs:line 35
I've Googled around, to not much avail. I've traced down the call chain, deep into the bowels of the SqlBulkCopy assembly (thanks, Resharper!), but the error seems to be hidden down deeper than I've been able to dig. I've tried installing (and loading) the appropriate SQL Server Types package (https://www.nuget.org/packages/Microsoft.SqlServer.Types/), but no dice.
Any suggestions?
回答1:
OK, I think I got it fixed. The problematic code was in the EFDataReader<T>
class (that I'd borrowed from https://github.com/MikaelEliasson/EntityFramework.Utilities/blob/master/EntityFramework.Utilities/EntityFramework.Utilities/EFDataReader.cs). Its GetValue(int ordinal)
originally looked like this:
public override object GetValue(int ordinal)
{
return Accessors[ordinal](Enumerator.Current);
}
But that meant that it was returning any db-agnostic DbGeometry
and DbGeography
values that happened to come through as DbGeometry
and DbGeography
, which the SqlBulkCopy class didn't understand. They actually need to be SQL-Server specific, i.e., SqlGeography
and SqlGeometry
, like so:
public override object GetValue(int ordinal)
{
object value = Accessors[ordinal](Enumerator.Current);
var dbgeo = value as DbGeography;
if (dbgeo != null)
{
var chars = new SqlChars(dbgeo.WellKnownValue.WellKnownText);
return SqlGeography.STGeomFromText(chars, dbgeo.CoordinateSystemId);
}
var dbgeom = value as DbGeometry;
if (dbgeom != null)
{
var chars = new SqlChars(dbgeom.WellKnownValue.WellKnownText);
return SqlGeometry.STGeomFromText(chars, dbgeom.CoordinateSystemId);
}
return value;
}
回答2:
FORWARD: I realize my expertise is not in
C#
yet, so I can only draw from my ownETL
experiences that is similar to your error. Likely, the problem may come down to your assumptions about the well-formed nature of the data and how this is being fed intoSQL
.
A trip to Spatial Data Types from MSDN informs us that the data needs to be well-formed....we knew that already...but have we assumed so on the source data?
You are using CSVLoader
, which comes from an external source, and in my own experience using SSIS
, data is not always structured correctly in the file. As mentioned, SQL Server
will balk at ill-formed spatial data types
that violate the columns datatype constraints.
- Have you qualified your data before using the call method?
- Are you using
spatial datatypes
that areinstantiable
? - Have you tried splitting the Bulk data up to test the consistency throughout the file(s) that
CSVLoader
retrieves them? Perhaps only part of your data is corrupt.
Since this is an integration operation, have you considered setting up a staging table to handle cleansing/transformation of your presumed well-formed data?
CSV
files are simple text files, so there is an implicit/explicit conversion between the CSVLoader
and when SQL Server
attempts to insert the rows by batch into the database. SQL Server
cannot violate the ACID
elements.
I cannot stress enough not to assume facts from your data and how C#
, let alone SQL Server
, reads and converts them. I have spent many an hour struggling with simple CSV
files in SSIS
before realizing that my IS
was parsing the file in a way that was unable to handle inconsistencies in the CSV
file (some of the data was corrupt or missing).
Hopefully, this will help you on your way to solving the problem.
Cheers,
来源:https://stackoverflow.com/questions/38407683/specified-type-is-not-registered-error-when-bulk-inserting-table-with-geospati