问题
Is there a way to create a job or azure service on Azure to move(cut) Azure blob files to Azure data lake store?
回答1:
I would say that the Azure Data Factory is a good fit for this. It supports scheduling (https://docs.microsoft.com/en-us/azure/data-factory/data-factory-scheduling-and-execution) and it supports the transfer of data from Blob to Azure Data Lake. See this example: https://docs.microsoft.com/en-us/azure/data-factory/data-factory-azure-datalake-connector
From the website:
Data Factory is a cloud-based data integration service that orchestrates and automates the movement and transformation of data. You can create data integration solutions using the Data Factory service that can ingest data from various data stores, transform/process the data, and publish the result data to the data stores.
Data Factory service allows you to create data pipelines that move and transform data, and then run the pipelines on a specified schedule (hourly, daily, weekly, etc.). It also provides rich visualizations to display the lineage and dependencies between your data pipelines, and monitor all your data pipelines from a single unified view to easily pinpoint issues and setup monitoring alerts.
https://docs.microsoft.com/en-us/azure/data-factory/data-factory-introduction
回答2:
Azure Data Lake Analytics is a computation service, not a data store.
Data Lake Analytics jobs can connect to data on:
- Data Lake Store
- Azure Storage (Blob): see https://docs.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-manage-use-portal#manage-account-data-sources
- Other sources through federated queries: http://sqlblog.com/blogs/jorg_klein/archive/2016/06/17/setup-azure-data-lake-analytics-federated-u-sql-queries-to-azure-sql-database.aspx
So you could either leave the data in blob storage, or use Azure Data Factory to transfer it to Data Lake Store to have best Analytics performance and enjoy its other benefits (see https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-overview).
来源：https://stackoverflow.com/questions/40995314/how-to-copy-azure-blob-files-to-azure-data-lake-analytics