Loading file from Azure Blob Storage into Azure SQL Database using BULK INSERT

Question

Following this announcement https://azure.microsoft.com/en-gb/updates/preview-loading-files-from-azure-blob-storage-into-sql-database/

I tried the example as such provided in this GitHub sample and receiving the following error,

-- Create credential with Azure Blob SAS
CREATE DATABASE SCOPED CREDENTIAL xxxstorcred
WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
SECRET = '?sv=2015-12-11&ss=bfqt&srt=sco&sp=rwdl&st=2017-03-14T17%3A52%3A00Z&se=2017-05-31T16%3A52%3A00Z&sig=f45435435TzrsdsdsdC8wonjDMBG0T0GJj717XetLEWReZ64eOrQE%3D';

-- Create external data source with with the roow URL of the Blob storage Account and associated credential.
CREATE EXTERNAL DATA SOURCE xxxstor
WITH (  TYPE = BLOB_STORAGE, 
        LOCATION = 'https://xxxstor.blob.core.windows.net', 
        CREDENTIAL= xxxstorcred);
--CREATE DESTINATION TABLE (if not exists)
DROP TABLE IF EXISTS Product;
GO

CREATE TABLE dbo.Product(
    Name nvarchar(50) NOT NULL,
    Color nvarchar(15) NULL,
    Price money NOT NULL,
    Size nvarchar(5) NULL,
    Quantity int NULL,
    Data nvarchar(4000) NULL,
    Tags nvarchar(4000) NULL
)
GO

--LOAD

-- INSERT CSV file into Product table
BULK INSERT Product
FROM 'random/product.csv' --random is the container name
WITH (  DATA_SOURCE = 'xxxstor',
        FORMAT='CSV', CODEPAGE = 65001, --UTF-8 encoding
        FIRSTROW=2,
        TABLOCK); 

Cannot bulk load because the file "random/product.csv" could not be opened. Operating system error code 1117(The request could not be performed because of an I/O device error.).

What am I missing?

Answer 1

I have tried the t-sql from the github sample that you provide. It works fine. From my test, there are two possibilities that will cause this error:

1) The container name is incorrect

2) the SAS SECRET is incorrect

Based on your description, I think your SAS secret is incorrect. Here is the secret that you used:

SECRET = '?sv=2015-12-11&ss=bfqt&srt=sco&sp=rwdl&st=2017-03-14T17%3A52%3A00Z&se=2017-05-31T16%3A52%3A00Z&sig=f45435435TzrsdsdsdC8wonjDMBG0T0GJj717XetLEWReZ64eOrQE%3D';

As my test, we need to remove '?'. Please try below secret:

SECRET = 'sv=2015-12-11&ss=bfqt&srt=sco&sp=rwdl&st=2017-03-14T17%3A52%3A00Z&se=2017-05-31T16%3A52%3A00Z&sig=f45435435TzrsdsdsdC8wonjDMBG0T0GJj717XetLEWReZ64eOrQE%3D';

For how to generate SAS, please refer to this article.

Answer 2

In my case it was a different problem with SAS Token: it was not valid yet.

I created a token with a default start date (which is the current time in my current time zone). But then it is being evaluated against GMT, which is behind my time.

You can easily test SAS token in Microsoft Azure Storege Explorer. In such case it will show a message like this:

The solution is to modify the start time e.g. to 00:00:00.