sharding

Entity Framework and sharded database

由 元气小坏坏 提交于 2019-12-09 12:59:53
问题 I have a WCF Data Service running on top of a Entity Framework code first 4.1 provider. The database is quite large (one key table has 77+ million records and grows by ~10% per month) and has presented quite a performance problem. While sharding a database that large is a pain it seems inevitable. My question is, has anybody successfully implemented EF with a sharded database and, if so, do you have any guidance? 回答1: Have you investigated the following options: Clustering your DB (I assume

NewSQL versus traditional optimization/sharding [closed]

由 爱⌒轻易说出口 提交于 2019-12-09 05:22:03
问题 As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. Closed 7 years ago . We're a small startup with a write-heavy SAAS app and are (finally!) getting to the point where our usage is presenting scaling issues

mongodb sharding replica sets with authentication via keyfile

由 ↘锁芯ラ 提交于 2019-12-08 10:00:37
问题 We are having two replica sets and one artiber node. In the replica set one is designated as primary and other as secondary. When one of them is shut down, automatically the other one takes over as primary. But under unforseen situations, when both the primary and secondary are down (also the arbiter) and when we restart them, the rs.status shows both as SECONDARY. None of them gets elected as primary. rs.status displays an error as below (NOTE: We tried changing the priority of pri/secondary

Can a list field be a shard key in MongoDB?

由 怎甘沉沦 提交于 2019-12-08 06:18:59
问题 Have some data that looks like this: widget: { categories: ['hair', 'nails', 'dress'] colors: ['red', 'white'] } The data needs to be queried like this: SELECT * FROM widget_table WHERE categories == 'hair' AND colors == 'red' Would like to put this data into a MongoDB sharded cluster. However, it seems like an ideal shard key would not be a list field. In this case, that is not possible because all of the fields are list fields. Is it possible to use a list field, such as the field

Integration of mongodb with elasticsearch in node.js environment

由 痴心易碎 提交于 2019-12-08 04:31:47
问题 I am working on a project which has following setup: I have an Amazon EC2 cluster with one master, 3 config servers and 3 shard servers. Master has a node.js application running which is basically a REST API written using Express.js module. I am using mongodb as database. Master has "mongos" service running which shards the data into 3 shard servers. Those servers have "mongod" service running on them. With this setup, I want to integrate elasticsearch to perform search queries. To do this I

Solrcloud multicore configuration

由 爱⌒轻易说出口 提交于 2019-12-07 13:33:43
问题 I have a standalone Solr instance with 4 different cores working fine using the embedded Jetty server. I configured the cores for v4.10.3 but since I moved to v5.1 and all seems to work fine without any changes. Before going into production, I need to set it up as a Solrcloud installation, initially with 2 nodes (two different machines) with 1 shard per node (to keep it simple). I have been trying to get it to work but I have not been able to do it. I tried to run it like this (I think using

Using multiple POSTGRES databases and schemas with the same Flask-SQLAlchemy model

由 怎甘沉沦 提交于 2019-12-07 07:30:52
问题 I'm going to be very specific here, because similar questions have been asked, but none of the solutions work for this problem. I'm working on a project that has four postgres databases, but let's say for the sake of simplicity there are 2. Namely, A & B A,B represent two geographical locations, but the tables and schemas in the database are identical. Sample model: from flask_sqlalchemy import SQLAlchemy from sqlalchemy import * from sqlalchemy.ext.declarative import declarative_base db =

django 1.4 database router - “cannot import name connection”

由 怎甘沉沦 提交于 2019-12-07 06:27:31
问题 I'm trying to shard my database on django, but I've got strange error on a first steps. I made a simple db router, that dous nothing: '''file /myproject/myapp/routers.py''' class ShardingRouter(object): def db_for_read(self, model, **hints): return 'default' def db_for_write(self, model, **hints): return 'default' def allow_relation(self, obj1, obj2, **hints): return 'default' def allow_syncdb(self, db, model): return 'default' I added in settings.py: DATABASE_ROUTERS = ['myproject.myapp

Sharding and ID generation as instagram

由 霸气de小男生 提交于 2019-12-07 01:42:29
My question is regarding ID generation for sharded environment. I am following the same steps as instagram did for generating unique ids. I have a few question on the implementation of this id generation in MySQL. This is how the ID is being generated (This is a PGQL stored procedure.) CREATE OR REPLACE FUNCTION insta5.next_id(OUT result bigint) AS $$ DECLARE our_epoch bigint := 1314220021721; seq_id bigint; now_millis bigint; shard_id int := 5; BEGIN SELECT nextval('insta5.table_id_seq') %% 1024 INTO seq_id; SELECT FLOOR(EXTRACT(EPOCH FROM clock_timestamp()) * 1000) INTO now_millis; result :=

Can a list field be a shard key in MongoDB?

由 此生再无相见时 提交于 2019-12-07 01:01:35
Have some data that looks like this: widget: { categories: ['hair', 'nails', 'dress'] colors: ['red', 'white'] } The data needs to be queried like this: SELECT * FROM widget_table WHERE categories == 'hair' AND colors == 'red' Would like to put this data into a MongoDB sharded cluster. However, it seems like an ideal shard key would not be a list field. In this case, that is not possible because all of the fields are list fields. Is it possible to use a list field, such as the field categories as the shard key in MongoDB? If so, what things should I look out for / be aware of? Thanks so much!
订阅 sharding