How can I empty a CosmosDB collection and fill it again and keep its metrics/logs

问题

we have a web service feed that we run daily and saves all documents into a CosmosDB collection, as there is no need for me to keep the old documents as the new feed comes in I am deleting and re creating the collection daily as well, this has some drawbacks

The statistics of the collection is reset so app insights and logging becomes useless
Its next to impossible to trouble shoot as all logs etc are also reset

How can I empty a CosmosDB collection before adding new documents to it so that all metrics etc are kept?

here is what I am doing currently

log.LogInformation("XXX--> Deleting Collection");
await docClient.DeleteDocumentCollectionAsync(collectionLink);

log.LogInformation("XXX--> Creating Collection");
defaultCollection = await docClient.CreateDocumentCollectionIfNotExistsAsync(databaseLink, defaultCollection, new RequestOptions { OfferThroughput = 1000 });

I want the same result but keeping all statistics etc.

回答1:

You can create a bulk delete stored procedure to delete all the documents from collection rather than deleting the collection.

A working implementation of such stored procedure can be found here: https://github.com/CosmosDB/labs/blob/3f49d8af44468ff7640cd3e382d13ba4c0299249/solutions/05-authoring_stored_procedures/bulk_delete.js

/**
 * A DocumentDB stored procedure that bulk deletes documents for a given query.<br/>
 * Note: You may need to execute this stored procedure multiple times (depending whether the stored procedure is able to delete every document within the execution timeout limit).
 *
 * @function
 * @param {string} query - A query that provides the documents to be deleted (e.g. "SELECT c._self FROM c WHERE c.founded_year = 2008"). Note: For best performance, reduce the # of properties returned per document in the query to only what's required (e.g. prefer SELECT c._self over SELECT * )
 * @returns {Object.<number, boolean>} Returns an object with the two properties:<br/>
 *   deleted - contains a count of documents deleted<br/>
 *   continuation - a boolean whether you should execute the stored procedure again (true if there are more documents to delete; false otherwise).
 */
function bulkDeleteProcedure(query) {
    var collection = getContext().getCollection();
    var collectionLink = collection.getSelfLink();
    var response = getContext().getResponse();
    var responseBody = {
        deleted: 0,
        continuation: true
    };

    // Validate input.
    if (!query) throw new Error("The query is undefined or null.");

    tryQueryAndDelete();

    // Recursively runs the query w/ support for continuation tokens.
    // Calls tryDelete(documents) as soon as the query returns documents.
    function tryQueryAndDelete(continuation) {
        var requestOptions = {continuation: continuation};

        var isAccepted = collection.queryDocuments(collectionLink, query, requestOptions, function (err, retrievedDocs, responseOptions) {
            if (err) throw err;

            if (retrievedDocs.length > 0) {
                // Begin deleting documents as soon as documents are returned form the query results.
                // tryDelete() resumes querying after deleting; no need to page through continuation tokens.
                //  - this is to prioritize writes over reads given timeout constraints.
                tryDelete(retrievedDocs);
            } else if (responseOptions.continuation) {
                // Else if the query came back empty, but with a continuation token; repeat the query w/ the token.
                tryQueryAndDelete(responseOptions.continuation);
            } else {
                // Else if there are no more documents and no continuation token - we are finished deleting documents.
                responseBody.continuation = false;
                response.setBody(responseBody);
            }
        });

        // If we hit execution bounds - return continuation: true.
        if (!isAccepted) {
            response.setBody(responseBody);
        }
    }

    // Recursively deletes documents passed in as an array argument.
    // Attempts to query for more on empty array.
    function tryDelete(documents) {
        if (documents.length > 0) {
            // Delete the first document in the array.
            var isAccepted = collection.deleteDocument(documents[0]._self, {}, function (err, responseOptions) {
                if (err) throw err;

                responseBody.deleted++;
                documents.shift();
                // Delete the next document in the array.
                tryDelete(documents);
            });

            // If we hit execution bounds - return continuation: true.
            if (!isAccepted) {
                response.setBody(responseBody);
            }
        } else {
            // If the document array is empty, query for more documents.
            tryQueryAndDelete();
        }
    }
}

来源：https://stackoverflow.com/questions/55699914/how-can-i-empty-a-cosmosdb-collection-and-fill-it-again-and-keep-its-metrics-log

标签

azure-cosmosdb