sharding

$lookup is new in MongoDB 3.2. It performs a left outer join to an unsharded collection in the same database to filter in documents from the “joined” collection for processing. To use $lookup , the from collection cannot be sharded. On the other hand, sharding is a useful horizontal scaling approach. What's the best practise to use them together? As the docs you quote indicate, you can't use $lookup on a sharded collection. So the best practice workaround is to perform the lookup yourself in a separate query. Perform your aggregate query. Pull the "localField" values from your query results

We've recently hit the >2 Million records for one of our main collections and now we started to suffer for major performance issues on that collection. They documents in the collection have about 8 fields which you can filter by using UI and the results are supposed to sorted by a timestamp field the record was processed. I've added several compound indexes with the filtered fields and the timetamp e.g: db.events.ensureIndex({somefield: 1, timestamp:-1}) I've also added couple of indexes for using several filters at once to hopefully achieve better performance. But some filters still take

I have an ES cluster with 4 nodes: number_of_replicas: 1 search01 - master: false, data: false search02 - master: true, data: true search03 - master: false, data: true search04 - master: false, data: true I had to restart search03, and when it came back, it rejoined the cluster no problem, but left 7 unassigned shards laying about. { "cluster_name" : "tweedle", "status" : "yellow", "timed_out" : false, "number_of_nodes" : 4, "number_of_data_nodes" : 3, "active_primary_shards" : 15, "active_shards" : 23, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 7 } Now my

为什么要分片？一般是数据库因业务的增长产生了巨大的压力情况下，开始是主从，然后是缓存，分区，均衡等，最后才会考虑的方案分片（一般是对主写入库进行操作），但，有经验的架构师应该从一开始就考虑到这种可能性，随着业务的增长可以线性的横向扩展数据库系统。在应用层开发时，预先考虑到以前架构要适应分片时的改造的可能性，在各方面进行优化，如sql尽量少用Join等。对于一量负载的网站，没必要很早就引入分片功能，这样反而使得业务逻辑更加复杂。 本文尝试给出一种用PHP实现的设计方案，shard_id的结构是64bit，10bit sharid系统+10bit为类型id+10子类型id+34bit自增id。关于分片设计的更多知识请参考我的其它文章。 <?php /** * 生产数据库分区id * * 描述 * * @package api * @author xxx * @copyright Copyright (c) 2014, xx.im. * @since Version 1.0 * @filesource * * @property database2 $database */ define( 'TOTAL_SHARD_NUM', 1 ); class shard { var $database = null; function shard() { $this->database =

What is the best approach for Sharding MySQL tables. The approaches I can think of are : Application Level sharding? Sharding at MySQL proxy layer? Central lookup server for sharding? Do you know of any interesting projects or tools in this area? The best approach for sharding MySQL tables to not do it unless it is totally unavoidable to do it. When you are writing an application, you usually want to do so in a way that maximizes velocity, developer speed. You optimize for latency (time until the answer is ready) or throughput (number of answers per time unit) only when necessary. You