Can we use Azure CLI to upload files to Azure Data Lake Storage Gen2

人盡茶涼 提交于 2020-02-02 16:22:09

问题


All I want to do, is to upload files from on prime to Azure Data Lake Storage Gen2 using the Azure CLI (via ` command), but have a connection error! Can I use Azure CLI to to that? Or I have to use another tool? PS: I cannot use Azure Data Factory, I want my job running from my on prime and not from the cloud! Thks.

azure.datalake.store.exceptions.DatalakeRESTException:  HTTP error: 
ConnectionError(MaxRetryError("HTTPSConnectionPool(host='storageAccount.azuredatalakestore.net', port=443): 
Max retries exceeded with url: /webhdfs/v1/my-file-system/data.csv?OP=GETFILESTATUS&api-version=2018-05-01 
(Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fc7ed169c50>: 
Failed to establish a new connection: [Errno -2] Name or service not known')

回答1:


No, Azure CLI for gen2 filesystem is not available, see this link.

If you want to upload file, here are two workarounds for you to refer.

  1. Use Azure Storage Explorer

  2. Use AzCopy v10, note only v10 supports Azure Data Lake Storage Gen2 APIs. Use myaccount.dfs.core.windows.net as a URI to call the ADLS Gen2 APIs.




回答2:


Got it work ^^ So the problem was with my authentication method, to make it work, you have to add your user as a Data Lake Storage Contributor + Owner. For anyone looking for the Role in the UI it's called "Storage Blob Data Contributor (Preview)". For a Resource Group choose Access Control (IAM) | Add in the blade locate the role Storage Blob Data Contributor (Preview) and assign access to the Users, Groups or Roles as meets your needs.



来源:https://stackoverflow.com/questions/54731300/can-we-use-azure-cli-to-upload-files-to-azure-data-lake-storage-gen2

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!