Mongoose: Cannot fetch file documents from .chunk collection without the data field

问题

I'm trying to delete the chunks from files that have not been fully uploaded to a MongoDB database.

When uploading files directly to MongoDB, the Mongoose, Multer & GridFSBucket packages seperate media file data into 2 sub-collections for the media collection: media.chunks & media.files.

In order to manually delete the file's chunks which are stored in the media.chunks collection, you need the file's ID, which is stored in both the media.files and the media.chunks collection (file metadata appears in the .files collection ONLY if the file has been fully uploaded). BUT, while incompletely uploaded files appear in the .chunks collection, they DO NOT appear in the .files collection! This means that though I can fetch fully uploaded file documents from the media.files collection in order to delete them using their IDs, I cannot do the same for incomplete files. Therefore, fetching file documents directly from the .chunks collection is NECESSARY in order to delete incomplete files.

While I can fetch the file documents from the .chunks collection, I cannot select just the file IDs from the collection. This causes the "Error: Maximum response size reached" error response since the document data is too large. It's also really slow when you fetch the data as well.

Here's my Node.js route to get all the file IDs from the .chunks sub-collection that does NOT work:

// Get the file IDs from the media.chunks collection.
router.get('/chunks', async function(req, res, next){
   
    var chunksQuery = await db.collection('media.chunks').find({},{data:0});    

    chunksQuery.toArray(function(error, docs) {
        console.log(docs)
        return res.json(docs);
    });
    
});

I have also tried with .select() but it doesn't work:

// Does NOT work.
var chunksQuery = await db.collection('media.chunks').find().select('-data');
// Does NOT work either.
var chunksQuery = await db.collection('media.chunks').find().select('-media.chunks.data');

回答1:

I believe I have figured this out. It is possible through the distinct() method:

// Get the distinct file IDs that correspond to each upload.
db.collection('media.chunks').distinct('files_id', function(error, results){
    console.log('Distinct file ID results',results);
    return res.status(200).send();
});

来源：https://stackoverflow.com/questions/64832155/mongoose-cannot-fetch-file-documents-from-chunk-collection-without-the-data-fi

标签

node.js

mongodb

mongoose

file-upload

gridfs