azure-data-lake

I am moving data into Azure Data Lake Store and processing it using Azure Data Lake Analytics. Data is in form of XML and I am reading it through XML Extractor . Now I want to access this data from Azure ML and it looks like Azure Data Lake store is not directly supported at the moment. What are the possible ways to use Azure Data Lake Store with Azure ML? Right now, Azure Data Lake Store is not a supported source, as you note. That said, Azure Data Lake Analytics can also be used to write data out to Azure Blob Store, and so you can use that as an approach to process the data in U-SQL and

I am working with integrating continous integration with vsts using MSBuild for U-SQL reffering the link : https://blogs.msdn.microsoft.com/azuredatalake/2017/10/24/continuous-integration-made-easy-with-msbuild-support-for-u-sql-preview/ But i am facing below issue while working with VSTS Build server Source\TrainingUsql\TrainingUsql\TrainingUsql\TrainingUsql.usqlproj (0, 0) Source\TrainingUsql\TrainingUsql\TrainingUsql\TrainingUsql.usqlproj(0,0): Error MSB4057: The target "Build" does not exist in the project. Process 'msbuild.exe' exited with code '1'. I have followed all the steps given in

I'm trying to build CI/CD for Azure Data lake analytics - USQL code and when i build the code using Visual studio build option in VSTS getting the below error - Using the Private agent for taking the build - C:\Users\a.sivananthan\AppData\Roaming\Microsoft\DataLake\MsBuild\1.0\Usql.targets(33,5): Error MSB4062: The "Microsoft.Cosmos.ScopeStudio.VsExtension.CompilerTask.USqlCompilerTask" task could not be loaded from the assembly Microsoft.Cosmos.ScopeStudio.VsExtension.CompilerTask. Could not load file or assembly 'Microsoft.Cosmos.ScopeStudio.VsExtension.CompilerTask' or one of its

Will USQL support to Compress and Decompress a file.? I would like decompress a compressed file to perform some validations and once they are passed, would like to compress the data to new file. In addition, doing automatic compression on OUTPUT is on the roadmap. Please add your vote to https://feedback.azure.com/forums/327234-data-lake/suggestions/13418367-support-gzip-on-output-as-well According to the main EXTRACT article, U-SQL EXTRACT method automatically recognises the GZip format, so you don't need to do anything special. Extraction from compressed data In general, the files are passed

I am receiving an error when using _adlsFileSystemClient.FileSystem.Create(_adlsAccountName, destFilePath, stream, overwrite) to upload files to a datalake. The error comes up with files over 30Mb. It works fine with smaller files. The error is: at Microsoft.Azure.Management.DataLake.Store.FileSystemOperations.d__16.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure