Read content of file from Data Lake Store using C# code

问题

I am trying to read the content from the Azure Data Lake Store file. But while connecting / opening the connection itself fails & exception is thrown

 var stream = _adlsFileSystemClient.FileSystem.Open(_adlsAccountName, "/folder1/"+file.PathSuffix);

Getting exception as:

Exception of type 'Microsoft.Rest.Azure.CloudException' was thrown.

Unable to recognize the issue. How to resolve this issue?

Thank you

回答1:

Exception of type 'Microsoft.Rest.Azure.CloudException' was thrown.

It is basic exception info. So please have a try to use fiddler tool to catch the detail error info.

According to your code, it seems that file is not found or authorization exception.

1.If it is 404 error: Please a try to use the following format and check the file from the azure portal.

 var srcPath = "/mytempdir/tomtest.txt";
 var stream = adlsFileSystemClient.FileSystem.Open(adlsAccountName, srcPath);

2.If it is the authorization exception and if you are using service-to-service authentication with client secret. Please assign permission to the azure AD Application to operate the file. More details please refer to the document.

The following is my test code and screenshot:

 var applicationId = "your application Id";
 var secretKey = "secret Key";
 var tenantId = "Your tenantId";
 var adlsAccountName = "adls account name";
 var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, applicationId, secretKey).Result;
 var adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds);
 var srcPath = "/mytempdir/tomtest.txt";
 var destPath = @"F:\tom\testfile.txt";
 using (var stream = adlsFileSystemClient.FileSystem.Open(adlsAccountName, srcPath))
 using (var fileStream = new FileStream(destPath, FileMode.Create))
 {
    stream.CopyTo(fileStream);
 }

Packages.config

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Hyak.Common" version="1.0.2" targetFramework="net452" />
  <package id="Microsoft.Azure.Common" version="2.1.0" targetFramework="net452" />
  <package id="Microsoft.Azure.Common.Dependencies" version="1.0.0" targetFramework="net452" />
  <package id="Microsoft.Azure.Management.DataLake.Store" version="1.0.4" targetFramework="net452" />
  <package id="Microsoft.Azure.Management.DataLake.StoreFileSystem" version="0.9.6-preview" targetFramework="net452" />
  <package id="Microsoft.Azure.Management.DataLake.StoreUploader" version="1.0.1-preview" targetFramework="net452" />
  <package id="Microsoft.Bcl" version="1.1.9" targetFramework="net452" />
  <package id="Microsoft.Bcl.Async" version="1.0.168" targetFramework="net452" />
  <package id="Microsoft.Bcl.Build" version="1.0.14" targetFramework="net452" />
  <package id="Microsoft.IdentityModel.Clients.ActiveDirectory" version="2.28.3" targetFramework="net452" />
  <package id="Microsoft.Net.Http" version="2.2.22" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime" version="2.3.5" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.5" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure.Authentication" version="2.2.12" targetFramework="net452" />
  <package id="Newtonsoft.Json" version="6.0.8" targetFramework="net452" />
</packages>

来源：https://stackoverflow.com/questions/42926878/read-content-of-file-from-data-lake-store-using-c-sharp-code

标签

azure

azure-data-lake