u-sql

问题 I'm using a custom OUTPUTTER to generate XML from my "flat data" like so: SELECT *.. OUTPUT @all_data TO "/patient/{ID}.tsv" USING new Microsoft.Analytics.Samples.Formats.Xml.XmlOutputter("Patient"); Which generates individual files that look like this: <Patient> <ID>5283293478</ID> <ANESTHESIA_START>09/06/2019 11:52:00</ANESTHESIA_START> <ANESHTHESIA_END>09/06/2019 14:40:00</ANESHTHESIA_END> <SURGERY_START_TIME>9/6/2019 11:52:00 AM</SURGERY_START_TIME> <SURGERY_END_TIME>9/6/2019 2:34:00 PM<

问题 UDT [SqlUserDefinedType(typeof(StudentHistoryFormatter))] public struct StudentHistory { public StudentHistory(int i, double? score, string status):this() { InstitutionId = i; Score = score; Status = status; } int InstitutionId { get; set; } double? Score {get; set; } string Status { get; set; } public string Value() { return string.Format("{0},{1},{2}", InstitutionId, Score, Status); } } For simplicity I did not even put the class in a namespace. I registered the assembly with the USQL

问题 I am trying to generate a dynamic FROM clause in U-SQL so that we can extract data from different files based on a previous query outcome. That's something like this: @filesToExtract = SELECT whatevergeneratesthepaths from @foo; <-- this query generates a rowset with all the file we want to extract like: [/path/file1.csv, /path/file2.csv] SELECT * FROM @filesToExtract; <-- here we want to extract the data from file1 and file2 I'm afraid that this kind of dynamics queries are not supported yet

问题 Situation : We have created database say "CLSTrackOMeter" and table say "Customer_Information" in Azure data lake Analytics. Customer_Information, stores the path of image in staging folder( For now i've hard code the source image path in class library). Agenda : use that value from CustInfo to upload data to Azure data lake store "Customer_Image" folder Tried Solution - Created usql class library, using .net sdk to upload files(Able to execute this class library in console application), and

问题 I am trying to Run an U-SQL script on Azure by C# code. Everything is created on azure (ADF, linkedservices, pipelines, data sets) after code gets executed but U-SQl script is not executed by ADF. I think there is an issue with startTime and end Time configured in pipeline code. I followed following article to complete this console application. Create, monitor, and manage Azure data factories using Data Factory .NET SDK Here is the URL of my complete C# code project for download. https://1drv

问题 How to standardize the output of USQL to have data for all the columns when converted from JSON We have a requirement to standardize the output of USQL. The USQL reads the JSON (source file) data and convert it to csv format. The problem is that the number of columns we have for each row in the csv is not the same because of missing data in the JSON. Sometimes the result set of USQL have a row in csv with "N" columns, another row is with "N+1" columns (cells). We would like to standardize the

问题 As these questions point out, Guid.NewGuid will return the same value for all rows due to the enforced deterministic nature of U-SQL i.e if it's scaled out if an element (vertex) needs retrying then it should return the same value.... Guid.NewGuid() always return same Guid for all rows auto_increment in U-SQL However.... the code example in the officials documentation for a User Defined Extractor purposefully uses Guid.NewGuid(). I'm not querying the validity of the answers for the questions

问题 I'm building CI/CD for my Azure Data lake Analytics - USQL code and facing below error while deploying my release using VSTS Power Shell task. "Access from 'example-app1' is denied. Please grant the user with necessary roles on Azure portal. Trace: 03e7229d-e7ca-43d5-a7be-6e0a3a3b9317" I have created Azure AAD following this link - https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal and created a service End point. I also gave access to

问题 I have a custom outputter for my USQL job which basically writes json output in a file using JsonTextWriter. I get following error when I try to compile error: "The type 'Uri' is defined in an assembly that is not referenced. You must add a reference to assembly 'System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'." at line: 60, column 20 Line number 60 is writer.WriteValue("Test"); I basically get this error for all the lines where I am doing WriteValue Here is my

问题 How do we generate a separate file for every record, containing a unique name? I would like every row in my dataset to have a unique identifier, a guid preferably, but it could be anything: @file = EXTRACT col1 string, col2 string, col3 string FROM @file1 USING Extractors.Csv(silent : true); @output = SELECT *, Guid.NewGuid().ToString() AS [myId] FROM @file; I would then create a separate file for each record: OUTPUT @output TO "/myFirstFunction_{myId}.txt" USING Outputters.Tsv(); The files