azure-data-factory

I am not sure whether Azure Data Factory project is supported on Visual Studio 2017 at the moment. I have just installed VS 2017 but cannot open our solution as there is one azure data factory project. Does Azure Data Factory support on Visual Studio 2017? I tweeted the Visual Studio team with exactly this issue a couple of days ago. I even included a screen shot very similar to yours. This is the response I got: @mrpaulandrew We have no info about it yet. Our team is already working on this. Please stay tuned to our blog: https://blogs.msdn.microsoft.com/visualstudio/ . So to answer your

Dynamically changing the connection string for Tablestorage or blob storage in Azure data factory. Currently, I could see such option for database related dataset? How to achieve the same in Table or Blob storage I believe this is what you wanted. https://docs.microsoft.com/en-us/azure/data-factory/parameterize-linked-services As doc mentioned, UI only supports 8 linked service. For others, you could change json code directly following the same pattern. { "name": "AzureBlobStorage12", "type": "Microsoft.DataFactory/factories/linkedservices", "properties": { "parameters": { "accountName": {

I have an Azure data factory (DF) pipeline that consists a Copy activity. The Copy activity uses HTTP connector as source to invoke a REST end-point and returns csv stream that sinks with Azure SQL Database table. The Copy fails when CSV contains strings (such as 40f52caf-e616-4321-8ea3-12ea3cbc54e9 ) which are mapped to an uniqueIdentifier field in target table with error message The given value of type String from the data source cannot be converted to type uniqueidentifier of the specified target column . I have tried to wrapped the source string with {} such as {40f52caf-e616-4321-8ea3

I have an ADF pipeline with copy activity, I'm copying data from blob storage CSV file to SQL database, this is working as expected. I need to map Name of the CSV file (this coming from pipeline parameters) and save it in the destination table. I'm wondering if there is a way to map parameters to destination columns. Column name can't directly use parameters. But you can use parameter for the whole structure property in dataset and columnMappings property in copy activity. This might be a little tedious as you will need to write the whole structure array and columnMappings on your own and pass

I have installed Powershell 6.0.0 on Ubuntu 16.04. I am getting following error when trying to create Azure Data Factory using Powershell Set-AzureRmDataFactoryV2 : The term 'Set-AzureRmDataFactoryV2' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:16 + $DataFactory = Set-AzureRmDataFactoryV2 -ResourceGroupName $ResGrp.Re ... + ~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (Set-AzureRmDataFactoryV2:String) [],

I have created 2 pipelines using the Copy data wizard, but neither show up when I click on the Diagram action. I get the message "This factory contains no pipelines or datasets". The pipelines both run successfully, and do show up when clicking on Pipelines blade. Both pipelines were configured as One time. Any ideas on why this is happening and how to fix it? The Monitor & Manage view for Azure Data Factory does not currently show the diagram for "run once" (aka oneTime or once-only) pipelines, as per here : Diagram view does not show one-time pipelines. This behavior is by design.

Hi i am currently trying to optimize an SSIS package when i do an Upsert/delete which handels about 93+ million rows from a table in a remote MariaDB source. The table also contains approximately 63 columns. Currently i'm using Sort and Merge Join in my package but as i've read some guides its recommended to do the sorting part in the server and not with the sort functionality in SSIS DT, as it puts a load on the SSIS Server Memory. And as i'm currently using this solution in Azure Data Factory running the package fails (most often Times out, even though i've increased the Time Out properties