key-value-store

Why Apache Kafka Streams uses RocksDB and if how is it possible to change it?

大兔子大兔子 提交于 2019-11-29 02:20:57
问题 During investigation within new features in Apache Kafka 0.9 and 0.10, we had used KStreams and KTables. There is interesting fact, that Kafka uses RocksDB internally. See Introducing Kafka Streams: Stream Processing Made Simple. RocksDB is not written in JVN compatible language, so it needs careful handling of the deployment, as it needs extra shared library (OS dependent). And here there are simple questions: Why Apache Kafka Streams uses RocksDB? How is it possible to change it? I had

Share a dict with multiple Python scripts

一世执手 提交于 2019-11-28 20:45:26
I'd like a unique dict (key/value) database to be accessible from multiple Python scripts running at the same time. If script1.py updates d[2839] , then script2.py should see the modified value when querying d[2839] a few seconds after. I thought about using SQLite but it seems that concurrent write/read from multiple processes is not SQLite's strength (let's say script1.py has just modified d[2839] , how would script2.py 's SQLite connection know it has to reload this specific part of the database? ) I also thought about locking the file when I want to flush the modifications (but it's rather

Pro's of databases like BigTable, SimpleDB

穿精又带淫゛_ 提交于 2019-11-28 16:52:36
问题 New school datastore paradigms like Google BigTable and Amazon SimpleDB are specifically designed for scalability, among other things. Basically, disallowing joins and denormalization are the ways this is being accomplished. In this topic, however, the consensus seems to be that joins on large tables don't necessarilly have to be too expensive and denormalization is "overrated" to some extent Why, then, do these aforementioned systems disallow joins and force everything together in a single

Lightweight Javascript DB for use in Node.js [closed]

南楼画角 提交于 2019-11-28 15:18:47
Anybody know of a lightweight yet durable database, written in Javascript, that can be used with Node.js. I don't want the 'weight' of (great) solutions like Mongo or Couch. A simple, in memory JS database with the capability to persist to disk as a file would be enough. I would only use it to store small amounts of data. Requirements: can run in process with a node.js server application can save the whole database to disk and recover after a failure NO need for atomic writes or transaction supports fast queries and sorting would be nice only needs to support small data volumes, up to 1MB in

Expressing multiple columns in berkeley db in python?

怎甘沉沦 提交于 2019-11-28 14:33:10
Say I have a simple table that contains username, firstname, lastname. How do I express this in berkeley Db? I'm currently using bsddb as the interface. Cheers. You have to pick one "column" as the key (must be unique; I imagine that would be "username" in your case) -- the only way searches will ever possibly happen. The other columns can be made to be the single string value of that key by any way you like, from pickling to simple joining with a character that's guaranteed to never occur in any of the columns, such as `\0' for many kind of "readable text strings". If you need to be able to

Need a MySQL query for selecting from a table storing key value pairs

泪湿孤枕 提交于 2019-11-28 12:35:57
I need to store few items and its properties in form of a key value pairs in the database (mySQL). I am planning to do it as following. I'll use two tables items and item_properties . items itemId | itemName ------------------- 1923 | AC 1235 | Fridge 8273 | Heater item_properties itemId | property | value -------------------------------- 1923 | effect | cooling 1923 | consumption | efficient 1923 | type | split 1235 | effect | cooling 1235 | volume | 20 liters 8273 | effect | heating 8273 | consumption | efficient 8273 | heatMethod | coil Now, if I have to select items whose 'effect' is

Storing object properties in redis

老子叫甜甜 提交于 2019-11-28 07:22:22
Lets say I have an object (User) which consists of a few properties (ID, Name, Surename, Age). Which way is better to store this object in redis? store each property value in dedicated key, for example user:{id}:id, user:{id}:name, user:{id}:surename, user:{id}:age store whole User object as JSON string in one key, for example user:{id}:json (value of the key will be something like this: {"ID": 123, "Name": "Johny", "Surename": "Bravo", "Age": 22}) According to these two sources probably the optimal solution would be to use hashes because of memory consumption when using dedicated keys and

What's the attraction of schemaless database systems?

时光怂恿深爱的人放手 提交于 2019-11-28 03:28:05
I've been hearing a lot of talk about schema-less (often distributed) database systems like MongoDB, CouchDB, SimpleDB, etc... While I can understand they might be valuable for some purposes, in most of my applications I'm trying to persist objects that have a specific number of fields of a specific type, and I just automatically think in the relational model. I'm always thinking in terms of rows with unique integer ids, null/not null fields, SQL datatypes, and select queries to find sets. While I'm attracted to the distributed nature and easy JSON/RESTful interfaces of these new systems, I

What scalability problems have you encountered using a NoSQL data store? [closed]

核能气质少年 提交于 2019-11-27 16:33:26
NoSQL refers to non-relational data stores that break with the history of relational databases and ACID guarantees. Popular open source NoSQL data stores include: Cassandra (tabular, written in Java, used by Cisco, WebEx, Digg, Facebook, IBM, Mahalo, Rackspace, Reddit and Twitter) CouchDB (document, written in Erlang, used by BBC and Engine Yard) Dynomite (key-value, written in Erlang, used by Powerset) HBase (key-value, written in Java, used by Bing) Hypertable (tabular, written in C++, used by Baidu) Kai (key-value, written in Erlang) MemcacheDB (key-value, written in C, used by Reddit)

Share a dict with multiple Python scripts

纵然是瞬间 提交于 2019-11-27 13:07:41
问题 I'd like a unique dict (key/value) database to be accessible from multiple Python scripts running at the same time. If script1.py updates d[2839] , then script2.py should see the modified value when querying d[2839] a few seconds after. I thought about using SQLite but it seems that concurrent write/read from multiple processes is not SQLite's strength (let's say script1.py has just modified d[2839] , how would script2.py 's SQLite connection know it has to reload this specific part of the