While downloading azure blobs to local file system an exception occurs

二次信任 提交于 2019-12-11 05:14:08

问题


While downloading azure blobs to local file system, I'm getting the following exception:

Client-Request-ID=99bdb0e4-2d1c-11e8-8fe6-00155dbf7128 Retry policy did not allow for a retry: Server-Timestamp=Wed, 21 Mar 2018 15:29:09 GMT, Server-Request-ID=1e7ab8f5-101e-0076-5329-c16a24000000, HTTP status code=404, Exception=The specified blob does not exist.
ErrorCode: BlobNotFound<?xml version="1.0" encoding="utf-8"?><Error><Code>BlobNotFound</Code><Message>The specified blob does not exist.
RequestId:1e7ab8f5-101e-0076-5329-c16a24000000Time:2018-03-21T15:29:09.6565984Z</Message></Error>.
Traceback (most recent call last):
  File "C:\Program Files\Commvault\ContentStore\Automation\CloudApps\CloudAppsUtils\cahelper.py", line 483, in download_contents_azure
    session.get_blob_to_path(container_name,fl,fl)
  File "C:\Program Files\Python36\lib\site-packages\azure\storage\blob\baseblobservice.py", line 1817, in get_blob_to_path
    timeout)
  File "C:\Program Files\Python36\lib\site-packages\azure\storage\blob\baseblobservice.py", line 2003, in get_blob_to_stream
    raise ex
  File "C:\Program Files\Python36\lib\site-packages\azure\storage\blob\baseblobservice.py", line 1971, in get_blob_to_stream
    _context=operation_context)
  File "C:\Program Files\Python36\lib\site-packages\azure\storage\blob\baseblobservice.py", line 1695, in _get_blob
    operation_context=_context)
  File "C:\Program Files\Python36\lib\site-packages\azure\storage\common\storageclient.py", line 354, in _perform_request
    raise ex
  File "C:\Program Files\Python36\lib\site-packages\azure\storage\common\storageclient.py", line 289, in _perform_request
    raise ex
  File "C:\Program Files\Python36\lib\site-packages\azure\storage\common\storageclient.py", line 275, in _perform_request
    HTTPError(response.status, response.message, response.headers, response.body))
  File "C:\Program Files\Python36\lib\site-packages\azure\storage\common\_error.py", line 111, in _http_error_handler
    raise AzureHttpError(message, http_error.status)
azure.common.AzureMissingResourceHttpError: The specified blob does not exist.ErrorCode: BlobNotFound
<?xml version="1.0" encoding="utf-8"?><Error><Code>BlobNotFound</Code><Message>The specified blob does not exist.
RequestId:1e7ab8f5-101e-0076-5329-c16a24000000
Time:2018-03-21T15:29:09.6565984Z</Message></Error>

I'm using the following code to download blobs:

def download_contents_azure(self, account_name, account_key, content):
    session=self.create_session_azure(account_name,account_key)
    os.mkdir('in place')
    os.chdir('in place')

    for item in content:
        # s = os.path.basename(item)

        path_to_file = ("/".join(item.strip("/").split('/')[1:]))
        container_name = Path(item).parts[1]
        gen = session.list_blobs(container_name)
        li = []

        for i in gen:
            li.append(i.name)

        if path_to_file in li:
            fl = os.path.basename(path_to_file)

            print(fl)
            c = self.splitall(item)

            for i in range(1,len(c)-1):
                if path.exists(c[i]) is False:
                    os.mkdir(c[i])
                os.chdir(c[i])

            session.get_blob_to_path(container_name,fl,fl)

            for i in range(1,len(c)-1):
                os.chdir("..")
        else:
            c = self.splitall(item)
            for i in range(1,len(c)):
                os.mkdir(c[i])
                os.chdir(c[i])

            generator = session.list_blobs(container_name,path_to_file+'/',delimiter='/')

            for blob in generator:
                bl = os.path.basename(blob.name)
                session.get_blob_to_path(container_name,bl,bl)

I've a path in azure (/container/folder/subfolder).
I'm trying to download the structure and all the files under subfolder. the first file under the subfolder gets downloaded and then I've the above exception. Due to this, I'm unable to loop through and print the next items.

Thoughts?


回答1:


Double check your line session.get_blob_to_path(container_name,fl,fl)

You have fl for blob_name.

documentation here indicates second argument is blob_name while third is file path on file system where to download.

I think your blob name is wrong. Hence it does not exist.

NOTE: consider installing fiddler so you can look at the network traces. It allows you to see the raw request.



来源:https://stackoverflow.com/questions/49410682/while-downloading-azure-blobs-to-local-file-system-an-exception-occurs

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!