azure-data-factory-2

问题 I have a for each activity which has a stored procedure (SP) wherein I am inputing values using item() evidently. Now suppose SP's input values are item().a, item().b and item().c Question: For some of the iteration of foreach, item().b does not exist which is expected. So how should i deal with it in the Stored procedure? Because at this point of time it is giving me an error when it executed SP by saying: "The template language expression 'item().b' cannot be evaluated because property 'b'

问题 We are using Azure data factory to copy data from on-premise to Azure. We have implemented multiple activities to complete the data copy. Until now, we are using basic authentication for web activity to call web API methods. As per the latest monitoring UI, it also supports MSI authentication. We have tried to use but no any luck. Also, tried to search related things but does not get any information related to data factory web activity and MSI authentication. How can we achieve this

问题 I am trying to copy data from Table storage to another Table storage of a different storage account, for that, I am using Copy activity in Azure data factory. I want to filter rows that are going to be copied to the sink table storage, for that Azure data factory gives an option to define a query. I want to apply a filter on the Partition key whose datatype is String but holds numeric values. I am looking at this documentation: https://docs.microsoft.com/en-us/azure/devops/pipelines/process

问题 Team. I have a situation to handle errors in ADF2 using Web Activity. Within a pipeline, any activity fails, it should trigger web activity. I attached the web activity after each activity's failed status. Therefore the web activity's dependsOn property looks like: "dependsOn": { "activity1": [ "Failed" ], "activity2": [ "Failed" ] } However, the web activity only gets fired when all activity1 and activity2 are failed. What I wanted was to get that fired either activity1 or activity2 is

问题 I've set up a Azure Data Factory pipeline containing a copy activity. For testing purposes both source and sink are Azure Blob Storages. I wan't to execute the pipeline as soon as a new file is created on the source Azure Blob Storage. I've created a trigger of type BlovEventsTrigger. Blob path begins with has been set to // I use Cloud Storage Explorer to upload files but it doesn't trigger my pipeline. To get an idea of what is wrong, how can I check if the event is fired? Any idea what

问题 How to standardize the output of USQL to have data for all the columns when converted from JSON We have a requirement to standardize the output of USQL. The USQL reads the JSON (source file) data and convert it to csv format. The problem is that the number of columns we have for each row in the csv is not the same because of missing data in the JSON. Sometimes the result set of USQL have a row in csv with "N" columns, another row is with "N+1" columns (cells). We would like to standardize the

问题 As we can run SSIS package on ADF-v2. So I've prepared an SSIS package to Sync records between Mysql and Azure SQL and getting below error in reports I'm tried using ODBC and ADO.Net connection but getting same result when trying executing from SSIS catalog. I'm able to sync records when i tried executing package from SSDT Question : Can't we use SSIS in ADF-v2, other than Azure cloud connecters? Also please suggest if there any steps i'm missing. Reference links used to implement same Link1

问题 I have a number of pipeline/linkedservice/dataset json files and I need to upload them to my Data Factory, opposed to creating new versions and copying the text over. Whats the simplest way to do this? 回答1: If you are using version 1, you can use Visual Studio to do so as shown here https://azure.microsoft.com/en-us/blog/azure-data-factory-visual-studio-extension-for-authoring-pipelines/ If you are using version 2, you can do this using powershell. First download and install the azure sdk for

问题 I was trying incremental sync to design incremental sync data between MySQL and Azure Sql referring article and while designing pipeline for new watermark found that lookup component only support SQL SERVER. Question Is there a way to sync incrementally on cloud from hosted Mysql to Azure SQl using ADF v1/v2? What other component we can use to select data from Mysql i tried Store Procedure but it support Sql server only. 回答1: We (ADF team) are actively working on expanding Lookup activity to

问题 failed with error Code: 2200 when retriving the data from rest api source to sql db in azure data factory. I have a rest api end point and i am connecting through azure data factory. Rest api returns the response in Json format and in different pages. each page contains 5000 records. i have achieved to retrieve the the pages by using foreach as shown in image. Rest api design in one date range i have 624 pages and the activity is sucessfull till 426 pages and upon started failing with below