Azure Data Factory Copy Activity - Can column mappings be composites or use a Templating function?

喜你入骨 提交于 2019-12-06 10:08:36

问题


When using a Data Factory Copy Activity to take delimited text files from Blob Storage to Table Storage, I am able to specify the input and output schemas and the relevant one-to-one Column Mappings.

It is possible to concatenate or format one or more columns (with $$Text.Format perhaps?) in the source schema and map these to a single column in the destination?

If possible, this would be especially useful if I can also set the Partition Key using a similar approach (a composite key taken from 2 or more source column values or better still, the SliceStart plus a concatenated source column value).


回答1:


I got a response from Microsoft on this and so thought I'd share:

At the moment you can't do this with an ADF Copy Activity. The suggested workaround is to have whatever data you need in the source text file in the correct format to start with (i.e. already concatenated into one column, or if you want a date to use as the slice date, then add this to the file as an additional column).

You can then map these columns directly to the Partition or Row Key as a one to one mapping.

This may require a pre-parse step of the delimited text files, which is best suited to a Custom .NET Activity in ADF upstream of the Blob file to Table storage step:

https://azure.microsoft.com/en-gb/documentation/articles/data-factory-use-custom-activities/




回答2:


Thanks for the question. Currently column mapping cannot use templating function, and you could request your wanted feature in this link: https://feedback.azure.com/forums/270578-azure-data-factory



来源:https://stackoverflow.com/questions/35324658/azure-data-factory-copy-activity-can-column-mappings-be-composites-or-use-a-te

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!