MongoDB Collection Structure Performance

问题

I have a MongoDB database of semi-complex records and my reporting queries are struggling as the collection size increases. I want to make some reporting Views that are optimized for quick searching and aggregating. Here is an sample format:

var record = {
fieldOne:"",
fieldTwo:"",
fieldThree:"", //There is approx 30 fields at this level
ArrayOne:[
    {subItem1:""},
    {subItem2:""} // There are usually about 10-15 items in this array
],
ArrayTwo:[
    {subItem1:""}, //ArrayTwo items reference ArrayOne item ids for ref
    {subItem2:""} // There are usually about 20-30 items in this array
],
ArrayThree:[
    {subItem1:""},// ArrayThree items reference both ArrayOne and ArrayTwo items for ref
    {subItem2:""},// There are usually about 200-300 items in this array
    {subArray:[
        {subItem1:""},
        {subItem2:""} // There are usually about 5 items in this array
    ]} 
]
};

I used to have this data where ArrayTwo was inside ArrayOne items and ArrayThree was inside ArrayTwo items so that referencing a parent was implied, but reporting became a nightmare with multiple nested levels of arrays.

I have a field called 'fieldName' at every level which is a way we target objects in the arrays.

I will often need to aggregate values from any of the 3 arrays across thousands of records in a query.

I see two ways of doing it.

A). Flatten and go Vertically, making a single smaller record in the database for every item in ArrayThree, essentially adding 200 records per single complex record. I tried this and I already have 200K records in 5 days of new data coming in. The benefit to this is that I have fieldNames that I can put indexing on.

B). Flatten Horizontally, making every array flat all within a single collection record. I would use the FieldName located in each array object as the key. This would make a single record with 200-300 fields in it. This would make a lot less records in the collection, but the fields would be dynamic, so adding indexes would not be possible(that I know of).

At this time, I have approx 300K existing records that I would be building this View off of. If I go vertical, that would place 60 Million simple records in the db and if I go Horizontal, it would be 300K records with 200 fields flattened in each with no indexing ability.

What's the right way to approach this?

回答1:

I'd be inclined to stick with the mongo philosophy and do individual entries for each distinct set/piece of information, rather than relying on references within a weird composite object.

60 Million records is "a lot" (but it really isn't "a ton"), and mongodb loves to have lots of little things tossed at it. On the flipside, you'd end up with fewer big objects and take up just as much space.

(*using the wired tiger back end with compression will make your disk go further too).

**edit: I'd also add that you really really really want indexes at the end of the day, so that's another vote for this approach.

来源：https://stackoverflow.com/questions/36018035/mongodb-collection-structure-performance