问题
I am trying to copy data from Table storage to another Table storage of a different storage account, for that, I am using Copy activity in Azure data factory.
I want to filter rows that are going to be copied to the sink table storage, for that Azure data factory gives an option to define a query. I want to apply a filter on the Partition key whose datatype is String but holds numeric values. I am looking at this documentation: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops there it says that type conversion is implicit for comparison operators like "eq", "le", "ge" etc
So if my query is "PartitionKey eq 0" it fails and gives this error:
A storage operation failed with the following error 'The remote server returned an error: (400) Bad Request.'.. Activity ID:edf8e608-d25e
But if I define my query as "PartitionKey eq '0'" it works.
I want to fetch rows with in the certain range of numbers for that I need to cast my partition key to a numeric value, How do I do that?
Also the "startsWith" and "endsWith" don't work
e.g, this query PartitionKey startsWith '10' gives the same error as above.
Looks like this: Thanks in advance.
回答1:
Firstly, to make sure that your query works - you can use Storage Explorer (preview) in Azure Portal to build the query in Query Builder mode:
and then switch to Text Editor:
Now, you are sure that you have got right query.
Let's apply this query to ADF. Without dynamic content - it will be exactly the same query:
In order to create a dynamic query - we need to add variables or parameters to define the boundary:
Afterward, create a dynamic content in query field, replacing query:
PartitionKey ge '0' and PartitionKey le '1'
with the following form using concat function:
@concat('PartitionKey ge ''0'' and PartitionKey lt ''1''')
Notice, that I must enquote single quote (') by adding extra one ('').
In the end - we need just to replace hard-coded values with previously defined parameters:
@concat('PartitionKey ge ''',pipeline().parameters.PartitionStart,''' and PartitionKey lt ''',pipeline().parameters.PartitionEnd,'''')
That's all. I hope that I explain how to achieve that by building dynamic content (query) in Azure Data Factory.
来源：https://stackoverflow.com/questions/56984020/azure-data-factory-expression-query-for-copy-activity