u-sql

U-SQL Output in Azure Data Lake

柔情痞子 提交于 2019-11-27 09:08:39
Would it be possible to automatically split a table into several files based on column values if I don't know how many different key values the table contains? Is it possible to put the key value into the filename? Michael Rys This is our top ask (and has been previously asked on stackoverflow too :). We are currently working on it and hopefully have it available by summer. Until then you have to write a script generator. I tend to use U-SQL to generate the script but you could do it with Powershell or T4 etc. Here is an example: Let's assume you want to write files for the column name in the

U-SQL - Extract data from json-array

纵然是瞬间 提交于 2019-11-26 20:46:53
问题 Already tried the suggested JSONPath option, but it seems the JSONExtractor only recognizes root level. In my case I have to deal with a nested json-structure, with an array as well (see example below). Any options for extracting this without multiple intermediate files? "relation": { "relationid": "123456", "name": "relation1", "addresses": { "address": [{ "addressid": "1", "street": "Street 1", "postcode": "1234 AB", "city": "City 1" }, { "addressid": "2", "street": "Street 2", "postcode":

Does U-SQL allow custom code to call external services

守給你的承諾、 提交于 2019-11-26 17:20:40
问题 In U-SQL custom code (code behind or Assemblies) can external services be called e.g. bing search or map. Thanks, Nasir 回答1: This is currently not supported for the following reason: Imagine that you write a UDF or UDO (e.g., an extractor) that calls a REST endpoint of a service that is used to get a few calls per minute from the same originating IP address. But now you execute this user code in a U-SQL job that is scaled out over millions of rows, running possibly on hundreds of vertices

U-SQL Output in Azure Data Lake

守給你的承諾、 提交于 2019-11-26 14:25:26
问题 Would it be possible to automatically split a table into several files based on column values if I don't know how many different key values the table contains? Is it possible to put the key value into the filename? 回答1: This is our top ask (and has been previously asked on stackoverflow too :). We are currently working on it and hopefully have it available by summer. Until then you have to write a script generator. I tend to use U-SQL to generate the script but you could do it with Powershell