gridfs

Object storage for a web application

你离开我真会死。 提交于 2019-12-01 18:59:56
I am currently working on a website where, roughly 40 million documents and images should be served to it's users. I need suggestions on which method is the most suitable for storing content with subject to these requirements. System should be highly available, scale-able and durable. Files have to be stored permanently and users should be able to modify them. Due to client restrictions, 3rd party object storage providers such as Amazon S3 and CDNs are not suitable. File size of content can vary from 1 MB to 30 MB. (However about 90% of the files would be less than 2 MB) Content retrieval

MongoDB - Maximum file size using GridFS

梦想的初衷 提交于 2019-12-01 18:56:55
I am wondering what is the maximum file size if someone wants to store a file using the GridFS? I can't find any information so any help would be appreciated. Infinity basically, or a more exact number would be how much your working set allows. The GridFS standard implemented within drivers will split files up into smaller chunks and store them in a fs.chunks collection irrespective of how big the file is. 来源: https://stackoverflow.com/questions/16604875/mongodb-maximum-file-size-using-gridfs

Display Image in GridFS

白昼怎懂夜的黑 提交于 2019-12-01 13:44:54
I am using gridfs-stream https://github.com/aheckmann/gridfs-stream & currently i am on displaying image from gridFS. When reading data it gives me following output. When i append this data to <img src="data:image/jpeg;base64,(data)"> , the image doesn't show. gfs // create a read stream from gfs... .createReadStream({ filename: 'error1.png' }) // and pipe it to Express' response .pipe(res); Output res :- Edited :- I tried this :- img.src = 'data:image/jpeg;base64,' + btoa(res); Output rendered is :- <img src="data:image/jpeg;base64,W29iamVjdCBPYmplY3Rd"> No image shown. I use file stream

Using MongoDB, Express, Node.Js and GridFS-stream for storing video and picture files

吃可爱长大的小学妹 提交于 2019-12-01 11:49:51
I am creating a single page application using JavaScript(JQuery) and need to store large video files which size exceed 16Mb. I found that need to use GridFS supporting large files. As I am the new one to MongoDB I am not sure how to use GridFS. There are some good tutorials on creating applications using Node.js, MongoDB and Express but cant find any describing how to use GridFS with MongoDB (not mongoose), Express and Node.js. I managed to put up stuff for uploading files in the BSON-document size limit of 16MB. This is what I have: var express = require('express'); var path = require('path')

分布式文件系统介绍

怎甘沉沦 提交于 2019-12-01 11:35:29
Google学术论文,这是众多分布式文件系统的起源 ================================== Google File System(大规模分散文件系统) MapReduce (大规模分散FrameWork) BigTable(大规模分散数据库) Chubby(分散锁服务) 一般你搜索Google_三大论文中文版(Bigtable、 GFS、 Google MapReduce)就有了。 做个中文版下载源:http://dl.iteye.com/topics/download/38db9a29-3e17-3dce-bc93-df9286081126 做个原版地址链接: http://labs.google.com/papers/gfs.html http://labs.google.com/papers/bigtable.html http://labs.google.com/papers/mapreduce.html GFS(Google File System) -------------------------------------- Google公司为了满足本公司需求而开发的基于Linux的专有分布式文件系统。。尽管Google公布了该系统的一些技术细节,但Google并没有将该系统的软件部分作为开源软件发布。 下面分布式文件系统都是类

nodejs display image stored in gridFS to html

余生长醉 提交于 2019-12-01 10:28:57
问题 Hi I'm new to nodejs and gridFS I'm trying to display images stored in gridFS to my html page Currently, I am using this code. gfs.exist(options, function(err, found){ if(err) return handleError(err); if(found) { console.log("Found Image"); var fs_write_stream = fs.createWriteStream('public/images/'+req.user._id + '_photo' + '.jpg'); var readstream = gfs.createReadStream({ filename: req.user._id + '_photo' }); readstream.pipe(fs_write_stream); readstream.on('close', function(){ console.log(

MongoDB GridFS File Sizes huge for relatively small file

不羁岁月 提交于 2019-12-01 09:40:36
问题 I'm doing some tests to see whether we can use GridFS on MongoDB to store files for a future application; I'm using 10gen's C# driver to "Upload" an 80Mb file onto the database. The first addition was fine and took approx 3 seconds which isn't too bad on my test machine; however future additions of the same file took much longer, up to 30 seconds eventually MongoDB told me it ran out of memory and crashed out. Adding 10 files, 80Mb in size results in 8 files being created for my database

how to download a file saved in gridFS using nodeJS

纵然是瞬间 提交于 2019-12-01 07:37:42
I need to download a resume from GridFS, below is the code ive written to do it, but this seems to not give me a physical file for download, this is used to reading the contents. How can i download the file? exports.getFileById = function(req, res){ var conn = mongoose.connection; var gfs = Grid(conn.db, mongoose.mongo); var id = req.params.ID; gfs.exist({_id: id,root: 'resume'}, function (err, found) { if (err) return handleError(err); if (!found) return res.send('Error on the database looking for the file.'); gfs.createReadStream({_id: id,root: 'resume'}).pipe(res); }); }; Hope this helps!

Using MongoDB, Express, Node.Js and GridFS-stream for storing video and picture files

泪湿孤枕 提交于 2019-12-01 07:34:13
问题 I am creating a single page application using JavaScript(JQuery) and need to store large video files which size exceed 16Mb. I found that need to use GridFS supporting large files. As I am the new one to MongoDB I am not sure how to use GridFS. There are some good tutorials on creating applications using Node.js, MongoDB and Express but cant find any describing how to use GridFS with MongoDB (not mongoose), Express and Node.js. I managed to put up stuff for uploading files in the BSON

Node.js displaying images from Mongo's GridFS

懵懂的女人 提交于 2019-12-01 01:25:38
I have a nodejs app that uses Mongo and GridFS to store images. I'm trying to display these images to the browser via Node.js (using express framework). I'm currently using: res.writeHead(200, {'Content-Type': 'image/jpeg' }); res.end(imageStore.currentChunk.data.buffer, 'binary'); imageStore is a the gridStore object after creating a new GridStore and calling gridStore.open(...) var gridStore = new GridStore(self.collection.db, doc._id, doc.filename, 'r', { chunk_size: doc.chunkSize }); gridStore.open(callback); I'm sure this isn't the right way, it displays a broken image. Any suggestions?