azure-data-factory

问题 I'm using ADF 2 and trying to grapple with the web activity. The tasks are. Copy file from blob storage and put the data in an azure SQL database Iterate through the data and use a PUT call to a REST API to update the data Okay so I can get the data in the table no problem. I can also make the call using a Web activity to the the API and put some hard coded data there. But I've been trying to use a For Each to iterate through the table I have and call the web activity to pass that data to the

问题 I am using Azure Data Lake Store (ADLS), targeted by an Azure Data Factory (ADF) pipeline that reads from Blob Storage and writes in to ADLS. During execution I notice that there is a folder created in the output ADLS that does not exist in the source data. The folder has a GUID for a name and many files in it, also GUIDs. The folder is temporary and after around 30 seconds it disappears. Is this part of the ADLS metadata indexing? Is it something used by ADF during processing? Although it

问题 In Azure Data Factory, how do I trigger a pipeline after other pipelines completed successfully? In detail: I seek to trigger an SSIS package after other pipelines completed successfully. I already know I can save my SSIS package as a pipeline and run it using a trigger like the other pipelines. But how do I make sure the SSIS package pipeline starts only after the other pipelines are finished? Is there a feature for this in Azure or do I need some kind of workaround for this? Thanks in

问题 Good day! I am working on moving files via Azure Data Factory from on-prem file store and/or ftp site to Azure Blob storage using Copy Data activity. When setting security access, I am using credential manager. However, when clicking 'Set credential' a string 'Preparing...' shows for a split moment, and then nothing happens and box is left blank. What is exactly credentials manager? Is is a separate application, which needs to be installed or Windows credentials manager available via

问题 I am trying to select some documents from documentDB collection, in an incremental way, so every slice will select based on the "timeCreated" field of the collection. The problem is that this field (timeCreated) is in seconds since the epoch (1970-01-01) and I could not find the proper format here. As project's assumptions, we are working with Azure Portal and without any programming interface, so the only solution I could think of is creating UDF in the DocumentDB that will transform the

问题 I have a azure data factory project. I need to query some data from my Azure SQL Database then load into an xml, zip it and upload to blob sotrage. I don't want to write anything to the file system (because I think the Azure Database doesn't have any lcoal storage) so I am using the Memorystream. This Script Task is working on my local SSIS database but not on the Azure Datafactory: using System; using System.Data; using Microsoft.SqlServer.Dts.Runtime; using System.Windows.Forms; using

问题 With Data Factory V2 I'm trying to implement a stream of data copy from one Azure SQL database to another. I have mapped all the columns of the source table with the sink table but in the sink table I have an empty column where I would like to enter the pipeline run time. Does anyone know how to fill this column in the sink table without it being present in the source table? Below there is the code of my copy pipeline { "name": "FLD_Item_base", "properties": { "activities": [ { "name": "Copy

问题 When I use data factory to update Azure ML models like the document said (https://docs.microsoft.com/en-us/azure/data-factory/v1/data-factory-azure-ml-update-resource-activity), I faced one problem: The blob reference: test/model.ilearner has an invalid or missing file extension. Supported file extensions for this output type are: ".csv, .tsv, .arff".'. I have searched the problem and found this solution: https://disqus.com/home/discussion/thewindowsazureproductsite/data_factory_create

问题 I am trying to add documents in CosmosDb that has a nested array. I am using the Copy Activity . Sample Document: { "itemNumber": "D10001", "readings" : [ { "value": 25, "ets":"100011111"}, { "value": 35, "ets":"100011122"} ] } In the source dataset I formatted the readings array as a string in my SQL query, and set the data type in the sink dataset as an Object . The data is copied, but the readings are stringified. Is there a means to configure the Copy Activity to handle this array? 回答1:

问题 I am copying data from a source, an API, and copying it into Azure SQL DB. But in one of the column I am getting Json objects. Any way i can use dynamic parameters (either through Pre-copy script or something else) in the pipeline to only take value of a particular tag from those json objects so that i can have only that value in the column. Only constraint is that I can't change the sink. It has to be Azure SQL DB. Json object I am getting: [{"self":"https://xxxxxxxx.jira.com/rest/api/2