Azure Blob - Read using Python

Question

Can someone tell me if it is possible to read a csv file directly from Azure blob storage as a stream and process it using Python? I know it can be done using C#.Net (shown

Answer 1

Yes, it is certainly possible to do so. Check out Azure Storage SDK for Python

from azure.storage.blob import BlockBlobService

block_blob_service = BlockBlobService(account_name='myaccount', account_key='mykey')

block_blob_service.get_blob_to_path('mycontainer', 'myblockblob', 'out-sunset.png')

You can read the complete SDK documentation here: http://azure-storage.readthedocs.io.

Answer 2

Here's a way to do it with the new version of the SDK (12.0.0):

from azure.storage.blob import BlobClient

blob = BlobClient(account_url="https://<account_name>.blob.core.windows.net"
                  container_name="<container_name>",
                  blob_name="<blob_name>",
                  credential="<account_key>")

with open("example.csv", "wb") as f:
    data = blob.download_blob()
    data.readinto(f)

See here for details.

Answer 3

One can stream from blob with python like this:

from tempfile import NamedTemporaryFile
from azure.storage.blob.blockblobservice import BlockBlobService

entry_path = conf['entry_path']
container_name = conf['container_name']
blob_service = BlockBlobService(
            account_name=conf['account_name'],
            account_key=conf['account_key'])

def get_file(filename):
    local_file = NamedTemporaryFile()
    blob_service.get_blob_to_stream(container_name, filename, stream=local_file, 
    max_connections=2)

    local_file.seek(0)
    return local_file

Answer 4

Provide Your Azure subscription Azure storage name and Secret Key as Account Key here

block_blob_service = BlockBlobService(account_name='$$$$$$', account_key='$$$$$$')

This still get the blob and save in current location as 'output.jpg'

block_blob_service.get_blob_to_path('you-container_name', 'your-blob', 'output.jpg')

This will get text/item from blob

blob_item= block_blob_service.get_blob_to_bytes('your-container-name','blob-name')

    blob_item.content

Answer 5

I recommend using smart_open.

from smart_open import open

# stream from Azure Blob Storage
with open('azure://my_container/my_file.txt') as fin:
    for line in fin:
        print(line)

# stream content *into* Azure Blob Storage (write mode):
with open('azure://my_container/my_file.txt', 'wb') as fout:
    fout.write(b'hello world')