azure-data-factory-2

问题 I have been following Microsoft's tutorial to incrementally/delta load data from an SQL Server database. It uses a watermark (timestamp) to keep track of changed rows since last time. The tutorial stores the watermark to an Azure SQL database using the "Stored Procedure" activity in the pipeline so it can be reused in the next execution. It seems overkill to have an Azure SQL database just to store that tiny bit of meta information (my source database is read-only btw). I'd rather just store

问题 This scenario is pretty straightforward, as described in ADFv2 docs and samples, I've created a copy pipeline to get the data from MongoDB collection, and write it to Azure SQL database. Full collection data is successfully transfered and all the mappings are set correctly. The problem starts when I try to filter the source dataset to get only the last n days from MongoDB. I've tried several queries, and cross-checked with MongoDB Compass to see if they're actually executing Mongo side, which

问题 I am using Azure Data Factory. I'm trying to use a String variable to lookup a Key in a JSON array and retrieve its Value. I can't seem to figure out how to do this in ADF. Details: I have defined a Pipeline Parameter named "obj", type "Object" and content: {"values":{"key1":"value1","key2":"value2"}} Parameter definition I need to use this pipeline to find a value named "key1" and return it as "value1"; "key2" and return it as "value2"... and so on. I'm planning to use my "obj" as a

问题 I am using Azure Data Factory. I'm trying to use a String variable to lookup a Key in a JSON array and retrieve its Value. I can't seem to figure out how to do this in ADF. Details: I have defined a Pipeline Parameter named "obj", type "Object" and content: {"values":{"key1":"value1","key2":"value2"}} Parameter definition I need to use this pipeline to find a value named "key1" and return it as "value1"; "key2" and return it as "value2"... and so on. I'm planning to use my "obj" as a

问题 I like to find the best methods of transferring 20 GB of SQL data from a SQL Server database installed on a customer onsite server, Client, to our Azure SQL Server, Source, on an S4 with 200 DTUs performance for $320 a month. When doing an initial setup, we set up an Azure Data Factory that copies over the 20 GB via multiple table copies, e.g., Client Table A's content to Source Table A, Client Table B's content to Source Table B, etc. Then we run many Extractors store procedures that insert

问题 How do we save data inside of an XML payload to blob storage? input <root> <alexIsAwesome>yes he is</alexIsAwesome> <bytes>sdfsdfjijOIJOISJDFQPWORPJkjsdlfkjlksdf==</bytes> </root> desired result <root> <alexIsAwesome>yes he is</alexIsAwesome> <bytes>/blob/path/toSavedPayload</bytes> </root> save bytes somewhere in blob replace bytes with URI of where bytes were saved How do we use data factory to extract a node from XML and save it to blob? 回答1: Currently, ADF doesn’t support XML natively.

问题 How do we save data inside of an XML payload to blob storage? input <root> <alexIsAwesome>yes he is</alexIsAwesome> <bytes>sdfsdfjijOIJOISJDFQPWORPJkjsdlfkjlksdf==</bytes> </root> desired result <root> <alexIsAwesome>yes he is</alexIsAwesome> <bytes>/blob/path/toSavedPayload</bytes> </root> save bytes somewhere in blob replace bytes with URI of where bytes were saved How do we use data factory to extract a node from XML and save it to blob? 回答1: Currently, ADF doesn’t support XML natively.

来源： https://stackoverflow.com/questions/54034373/adf-log-analytics-how-to-correlate-multiple-pipelines

来源： https://stackoverflow.com/questions/54034373/adf-log-analytics-how-to-correlate-multiple-pipelines

来源： https://stackoverflow.com/questions/54034373/adf-log-analytics-how-to-correlate-multiple-pipelines