nosql

HBase: 基本介绍

点点圈 提交于 2019-12-10 19:17:31
HBase基本介绍 简介 : hbase是bigtable的开源java版本。是 建立在hdfs之上 ,提供高可靠性、高性能、列存储、可伸缩、实时读写nosql的 数据库系统。 它介于nosql和RDBMS之间,仅能通过主键(row key)和主键的range来检索数据,仅支持单行事务(可通过hive支持来实现多表join等复杂操作)。 主要用来存储结构化和半结构化的松散数据。 Hbase查询数据功能很简单,不支持join等复杂操作,不支持复杂的事务(行级的事务) Hbase中支持的数据类型:byte[] 与hadoop一样,Hbase目标主要依靠横向扩展,通过不断增加廉价的商用服务器,来增加计算和存储能力。 HBase中的表一般有这样的特点 : 大:一个表可以有上十亿行,上百万列 面向列:面向列(族)的存储和权限控制,列(族)独立检索 稀疏:对于为空(null)的列,并不占用存储空间,因此,表可以设计的非常稀疏 传统数据表- HBase的发展历程 HBase的原型是Google的BigTable论文,受到了该论文思想的启发,目前作为Hadoop的子项目来开发维护,用于支持结构化的数据存储。 官方网站:http://hbase.apache.org 2006年Google发表BigTable白皮书 2006年开始开发HBase 2008 HBase成为了 Hadoop的子项目

Mongoengine PointField gives location object expected, location array not in correct format error

試著忘記壹切 提交于 2019-12-10 19:12:30
问题 I have a model as follows: class Station(Document): location = PointField() Trying to write data as follows: station = Station() station.location = { "type": "Point", "coordinates": [ 81.4471435546875, 23.61432859499169 ] } station.save() However this gives the error Could not save document (location object expected, location array not in correct format) Mongoengine documentation says such a dictionary should be OK. What am I missing here? 回答1: I face similar problem once, in my case it

【 Redis 详解 】------ 介绍、配置 、优化

|▌冷眼眸甩不掉的悲伤 提交于 2019-12-10 18:40:33
文章目录 一、了解关系数据库(SQL)和非关系数据库(NoSQL): (1)关系型数据库(SQL): (2)非关系型数据库(NoSQL): (3)非关系型数据库产生的背景: 二、Redis 简介: 三、安装部署 Redis: (1)安装 redis: (2)redis-cli 远程连接: 四、Redis 配置文件: 五、Redis 数据库常用命令: (一)常用命令: (二)Redis 多数据操作: 六、Redis 持久化: (1)持久化概述: (2)持久化分类: (3)RDB持久化: (4)AOF 持久化: 七、Redis 性能管理: (1)内存碎片率: (2)内存使用率: (3)回收 Key: 一、了解关系数据库(SQL)和非关系数据库(NoSQL): (1)关系型数据库(SQL): 1、一个结构化的数据库,创建在关系模型基础上,一般面向于记录; 2、包括 Oracle 、Mysql 、 SQL Server 、Microsoft Access 、DB2 等。 优点: 1、易于维护:都是使用表结构,格式一致; 2、使用方便:SQL语言通用,可用于复杂查询; 3、复杂操作:支持SQL,可用于一个表以及多个表之间非常复杂的查询。 缺点: 1、读写性能比较差,尤其是海量数据的高效率读写; 2、固定的表结构,灵活度稍欠; 3、对于高并发读写需求,传统关系型数据库来说,硬盘I

springboot2 log4j 2 配置Mongodb3

家住魔仙堡 提交于 2019-12-10 18:14:00
【推荐】2019 Java 开发者跳槽指南.pdf(吐血整理) >>> 第一步、查看正经的配置保证该有的引入包一致 <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> <!--使用log4j2需要去除loggin--> <exclusions> <exclusion> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-logging</artifactId> </exclusion> </exclusions> </dependency> <!--log4j2--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-log4j2</artifactId> </dependency> <!-- https://mvnrepository.com/artifact/org.mongodb/mongo-java-driver --> <dependency> <groupId>org.mongodb<

How to get doc from couchdb without some fields? using Ektorp if possible

二次信任 提交于 2019-12-10 18:12:33
问题 At first I misunderstood my problem and posted this question : Can someone explain me Cascading and FetchType lazy in Ektorp? What I need to do: I need to save an Entity in couchdb and then have way to read it and potentially ignore some fields. So I came up with this solution: I create a show function that delete fields from an object and then send it back. function(doc, req) { var result = doc; var ignore = JSON.parse(decodeURIComponent(req.query.ignore)); //this is an array of field names

NoSql solution to store 20[TB] of data, as vector/array?

橙三吉。 提交于 2019-12-10 18:11:14
问题 I need to build a system to efficiently store & maintain a huge amount (20 [TB]) of data (and be able to access it in 'vector' form). Here are my dimensions: (1) time (given as an integer of the form YYYYMMDDHHMMSS) (2) field (a string of any given length, representing a name of a hospital) (3) instrumentID (an integer representing a uniqueID for the instrument) I will need a way to be able to store data individually, meaning, something like: STORE 23789.46 as the data for instrumentID = 5 on

Storing JS arrays and objects in a database

丶灬走出姿态 提交于 2019-12-10 18:07:23
问题 I have an application that lets users build things in JS. I want the user to be able to save the current state of his work to reuse it or share it, but what he has is a collection of JS objects stored in a JS array, with very different properties (color, label, x/y position, size, etc.). SQL seems terrible for that particular task, forcing me to maintain tables for every different object, and alas I know very little about NoSQL database. What tools would you use to perform this ? MongoDB

Easiest way to import a simple csv file to a graph with OrientDB ETL

纵饮孤独 提交于 2019-12-10 17:47:40
问题 I would like to import a very simple directed graph file in csv to OrientDB. Concretely, the file is the roadNet-PA dataset from the SNAP collection https://snap.stanford.edu/data/roadNet-PA.html. The first lines of the file are as follows: # Directed graph (each unordered pair of nodes is saved once) # Pennsylvania road network # Nodes: 1088092 Edges: 3083796 # FromNodeId ToNodeId 0 1 0 6309 0 6353 1 0 6353 0 6353 6354 There is only one type of vertex (a road intersection) and edges have no

MongoDB full text search and lookup operator [duplicate]

南笙酒味 提交于 2019-12-10 17:12:33
问题 This question already has an answer here : How to perform a $text search on a 'joined' collection via $lookup? (1 answer) Closed last month . Full text search in MongoDB seems to be a nice feature, especially when it comes to the need of high performance search and indexes. However, I am wondering why a full text search is not allowed over more than one collection. For example: I have a supplier and a products collection (every supplier has n products) I would really like to do a search over

数据库Mongodb

冷暖自知 提交于 2019-12-10 16:48:54
1.数据库 1.什么是数据库? 存储数据的一个仓库 2.数据库的分类? 关系型数据库( SQL )=>( mysql )和非关系型数据库( NOSQL )=>( Mongodb ) 3.如何区分一个数据库属于哪个类别? 看有没有表,有表就是 SQL ,没有就是 NOSQL 4.数据库的构成? 关系型数据库 数据库 -> 表 -> 行 非关系型数据库 数据库 -> 集合 -> 文档 2. MongoDB 1.概念 基于分布式文件存储的数据库 2.存储类型: BSON bson 是 json 的超集, bson 拥有 json 的所有,但是它有 json 没有的二进制类型 3. MongoDB 操作命令 show dbs 查看数据库列表 use 数据库名称 创建或是切换数据库 如果这个数据库没有,那么就是创建 如果有,那就是切换数据库 db.集合名称[ 复数 ].save() // 创建一个集合,并且往这个集合添加一条数据 db.集合名称[ 复数 ].insert() // 创建一个集合,并且往这个集合添加一条数据 save如果指定主键,那么就是修改 insert指定主键添加,会报错 show collections 查看数据库下有几个集合 db.users.remove({}) 删除数据库 db.users.find() 查找users集合中所有数据 db.users.findOne