azure-cosmosdb

How to improve the performance when copying data from cosmosdb?

北慕城南 提交于 2019-12-11 05:28:31
问题 I am now trying to copy data from cosmosdb to data lake store by data factory. However, the performance is poor, about 100KB/s, and the data volume is 100+ GB, and keeps increasing. It will take 10+ days to finish, which is not acceptable. Microsoft document https://docs.microsoft.com/en-us/azure/data-factory/data-factory-copy-activity-performance mentioned that the max speed from cosmos to data lake store is 1MB/s. Even this, the performance is still bad for us. The cosmos migration tool

Microsoft Azure CosmostDB Script Explorer console.log

孤街醉人 提交于 2019-12-11 05:27:11
问题 I am trying to debug a stored procedure or a script written in Microsoft Azure CosmosDB 'Script Explorer' using javascript. I put several console.log() messages so that I could trace my proc and I couldn't find where those log messages are written to. On the script frame, there is a text box to enter your input variables and below that there is a text box which displays response. Where do I find the log messages that I log using console.log()? thanks. 回答1: I've found a getScriptLog method to

Saving & Testing Stored Procedures/Triggers (maybe User Defined Functions) For Partitioned Collections

杀马特。学长 韩版系。学妹 提交于 2019-12-11 05:19:04
问题 I'm receiving the following error when attempting save modifications to a Stored Procedure that has been created within a partitioned collection: Failed to save the script Here is the details from within the Azure Portal: Operation name Failed to save the script Time stamp Fri Feb 17 2017 08:46:32 GMT-0500 (Eastern Standard Time) Event initiated by - Description Database Account: MyDocDbAccount, Script: bulkImport, Message: {"code":400,"body":"{\"code\":\"BadRequest\",\"message\":\" Replaces

Indexing on nested field in Azure Cosmos DB

做~自己de王妃 提交于 2019-12-11 05:07:47
问题 I want to create an index on a nested field in a document in Azure Cosmos DB. E.g. if I have the following schema: { 'id': 1, 'nested': { 'mode': 'mode1', 'text': 'nice text' } } I want to create an index on the field nested.mode . How can this be done? 回答1: By default, all paths are indexed in Cosmos DB. To index just "nested.mode", you need to specify an indexing policy on the /nested/mode/? path with the appropriate data type/precision. Something like this in JSON within the includedPaths

DocumentDB Query Requires Unexpected High RUs

怎甘沉沦 提交于 2019-12-11 04:50:32
问题 I have about 200MB of data stored in a S3 document collection (the highest one). Each document is about 300KB in size. But when I ran my query, I am surprised to see that it requires 7245.81 RUs. Because S3 is at 2500 RUs/Second, this performance won't scale for my application. I just want to ask if I am doing anything wrong and if there is anything that I can do to improve it? My query is like this: SELECT item.Id,item.Priority, va.something, wa.something... 11 fields in total FROM model

filtering DocumentDb collection from Data Factory using unix-timestamp

青春壹個敷衍的年華 提交于 2019-12-11 03:54:55
问题 I am trying to select some documents from documentDB collection, in an incremental way, so every slice will select based on the "timeCreated" field of the collection. The problem is that this field (timeCreated) is in seconds since the epoch (1970-01-01) and I could not find the proper format here. As project's assumptions, we are working with Azure Portal and without any programming interface, so the only solution I could think of is creating UDF in the DocumentDB that will transform the

CosmosDB Join (SQL API)

*爱你&永不变心* 提交于 2019-12-11 03:37:40
问题 I'm using CosmosDB using SQL API and I'm trying to join two collections. I saw join example within a document but not getting what actually looking. RequestLog { "DateTimeStamp": "2018-03-16T10:56:52.1411006Z", "RequestId": "8ce80648-66e2-4357-98a8-7a71e8b65301", "IPAddress": "0.0.0.173" } ResponseLog { "DateTimeStamp": "2018-03-16T10:56:52.1411006Z", "RequestId": "8ce80648-66e2-4357-98a8-7a71e8b65301", "Body": "Hello" } Is it possible to join both collections? how? 回答1: Actually Cosmos DB

How to update a document in Documentdb using queries?

大憨熊 提交于 2019-12-11 03:26:02
问题 How to update a document in Document db using queries ( basically want to update a document using a stored procedure)? 回答1: The following sample might be what you need: https://github.com/aliuy/documentdb-serverside-js/blob/master/stored-procedures/update.js. Here's a simplified version: function updateSproc(id, update) { var collection = getContext().getCollection(); var collectionLink = collection.getSelfLink(); var response = getContext().getResponse(); tryQueryAndUpdate(); function

DocumentDB - query result order

∥☆過路亽.° 提交于 2019-12-11 03:21:53
问题 I am using Azure DocumentDB to store information about users. These documents contain a property called date_created . I'd like to send a query to documentDB to retrieve ALL users stored within this collection. I also want the result to be ordered from newest to oldest. I currently implemented a stored procedure in DocumentDB that goes through the collection and sorts the results based on the date property. However, I have 2 questions: When returning multiple documents for a query, what is

Error inserting a document in a MongoDB sharded collection created with the CLI

南笙酒味 提交于 2019-12-11 02:42:31
问题 Using the latest azure cli (2.0.21) I am trying to create in CosmosDB a sharded MongoDB collection with the following script (from powershell) : $resourceGroupName="RG-S01-APP-CASRWA-001" $name="marcoolandesetest" $databaseName="marco" $collectionName="fromcli" $originalThroughput=100000 $kind="MongoDB" $partition="/TaskId" az login az cosmosdb create --name $name --kind $kind --resource-group $resourceGroupName az cosmosdb database create --name $name --db-name $databaseName --resource-