nosql

MongoDb speed decrease

喜你入骨 提交于 2019-12-11 04:17:28
问题 I use mongodb to store compressed html files . Basically, a complete document of mongod is like: {'_id': 1, 'p1': data, 'p2': data2, 'p3': data3} where data, data1, data3 are : bson.binary.Binary(zlib_compressed_html) I have 12 Million ids and dataX are each one average 90KB, so each document has at least size 180KB + sizeof(_id) + some_overhead . The total data size would be at least 2TB. I would like to notice that '_id' is index. I insert to mongo with the following way: def _save(self,

Convert query builder conditions to MongoDB operations including nested array of subdocuments

无人久伴 提交于 2019-12-11 03:36:40
问题 I am building an application in Angular 8 on the client side and NodeJS 12 with MongoDB 4 / Mongoose 5 on the server side. I have a query generated by the Angular2 query builder module. The Angular query builder object is sent to the server. I have a server-side controller function that converts the Angular query object to MongoDB operations. This is working perfectly for generating queries for top-level properties such as RecordID and RecordType . This is also working for building nested and

Cant update Dynamo Db table , getting ValidationException

橙三吉。 提交于 2019-12-11 03:24:48
问题 I need to update my dynamo db table by using only partition key. But i got validation exeption. I have created a table with 3 fields. id (Partition Key) name (Sort Key) age Then i have triyed to update age field using only id.(tryied to modify age 30 to 40) this is my code var AWS = require("aws-sdk"); AWS.config.update({ region: "us-east-1", }); var params = { TableName: 'test', Key: { id: '100' }, UpdateExpression: 'set #age = :age ', ConditionExpression: '#age = :testAge',

mongodb - Finding the Sum of a field (if it exists) in a collection

一个人想着一个人 提交于 2019-12-11 03:14:12
问题 I have the following mongodb structure { "_id": ObjectId("507c80a143188f9610000003"), "date": ISODate("2012-10-15T21: 31: 13.0Z"), "uid": NumberInt(35920), "comp": NumberInt(770), "fields": { "rating": { "parent": "rating", "weight": NumberInt(2), "rel_weight": 0.11, }, "capacity": { "parent": "capacity", "weight": NumberInt(4), "rel_weight": 0.89, }, } } The "fields" attribute has 2 fields "rating" and "capacity" in it. But, each entry might have a different set of fields. eg. dimension,

Google Datastore - Not Seeing 1 Write per Second per Entity Group Limitation

别说谁变了你拦得住时间么 提交于 2019-12-11 03:12:49
问题 I've read a lot about strong vs eventual consistency, using ancestor / entity groups, and the 1 write per second per entity group limitation of Google Datastore. However, in my testing I have never hit the exception Too much contention on these datastore entities. please try again. and am trying to understand whether I'm misunderstanding these concepts or missing a piece of the puzzle. I'm creating entities like so: func usersKey(c appengine.Context) *datastore.Key { return datastore.NewKey(c

Firebase retrieve data under certain child

时光总嘲笑我的痴心妄想 提交于 2019-12-11 03:09:45
问题 I was having some trouble when trying to retrieve from firebase. Here is my firebase structure: What I tried to do is firstly, I wanted to get list of receiptItemID in the first picture. Then, after I get the IDs, for each ID, I wanted to get its quantity and type. After that, I will store them into array and perform some sorting. Here is my code: var query = firebase.database().ref(''); query.once( 'value', data => { data.forEach(subtypeSnapshot => { var itemKey = subtypeSnapshot.key; var

MongoDB - How can I use multiple groups in an aggregation pipeline?

吃可爱长大的小学妹 提交于 2019-12-11 02:53:00
问题 I am trying to perform some aggregation on a collection of documents. The documents look like this: {name: "Sara", brandId: 1, count: 2 day:1/6/2014}//day value is ISODate {name: "Sally", brandId: 1, count: 5 day:1/7/2014} {name: "Mike", brandId: 1, count: 2, day: 1/8/2014} {name: "Bob", brandId: 1, count: 4 day: 1/8/2014} {name: "Joe", brandId: 1, count: 5 day:1/8/2014} What I am trying to do is get the aggregate sum of all the 'count' values. Then I would like to group the documents by a

MongoDb aggregation

帅比萌擦擦* 提交于 2019-12-11 02:46:30
问题 How to write simial query using mongdoDb aggregation select count(*) as ccount from a group by a.someField order by ccount desc limit 1 Using group function in mondoDb, I want to group my collection by specific key and return 1 row, which occurs more often Code below in Java, return list, that was apply group by on field someField and return collection of elements, where every element has 2 fields, "someField" and "count" (number of occurrence) I would like to avoid second iteration on client

Updating nested document in MongoDB

China☆狼群 提交于 2019-12-11 02:45:14
问题 I'm trying to update a hash that's nested in a MongDB document. I know about the $push function for arrays, and $set for completely overwriting elements, but I can't quite get the behaviour I'm looking for. Here's what I'm trying to get: Before: { 'id' => 1234, 'evaluators' => { 'A' => { 'x' => 2, 'y' => 4 }, } } Expected, after: { 'id' => 1234, 'evaluators' => { 'A' => { 'x' => 2, 'y' => 4 }, 'B' => { 'x' => 3, 'y' => 5 }, } } I've tried doing (in Ruby): coll.update({ :id => id }, { '$set' =

AWS DynamoDB read after write consistency - how does it work theoretically?

扶醉桌前 提交于 2019-12-11 02:36:51
问题 Most of the nosql solution only use eventually consistency, and given that DynamoDB replicate the data into three datacenter, how does read after write consistency is being maintained? What would be generic approach to this kind of problem? I think it is interesting since even in MySQL replication data is replicated asynchronously. 回答1: I'll use MySQL to illustrate the answer, since you mentioned it, though, obviously, neither of us is implying that DynamoDB runs on MySQL. In a single network