rds

最佳实践 | RDS & POLARDB归档到X-Pack Spark计算

生来就可爱ヽ(ⅴ<●) 提交于 2019-12-04 06:43:09
X-Pack Spark服务通过外部计算资源的方式,为Redis、Cassandra、MongoDB、HBase、RDS存储服务提供复杂分析、流式处理及入库、机器学习的能力,从而更好的解决用户数据处理相关场景问题。 RDS & POLARDB分表归档到X-Pack Spark步骤 一键关联POLARDB到Spark集群 一键关联主要是做好spark访问RDS & POLARDB的准备工作。 POLARDB表存储 在database ‘test1’中每5分钟生成一张表,这里假设为表 'test1'、'test2'、'test2'、... 具体的建表语句如下: *请左右滑动阅览 CREATE TABLE `test1` ( `a` int(11) NOT NULL, `b` time DEFAULT NULL, `c` double DEFAULT NULL, PRIMARY KEY (`a`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 归档到Spark的调试 x-pack spark提供交互式查询模式支持直接在控制台提交sql、python脚本、scala code来调试。 1、首先创建一个交互式查询的session,在其中添加mysql-connector的jar包。 *请左右滑动阅览 wget https://spark-home.oss-cn

AWS: What can I use to run periodic tasks on RDS?

此生再无相见时 提交于 2019-12-04 06:29:18
In specific RDS column as a date, I keep the information when user's trials end. I'm going to check everyday these dates in database and when less the few days lefts to the end of trial, I want send an email message (with SES). How can I run a periodic tasks in AWS to check database? I know that I can use: Lambda EC2 (or Elastic Beanstalk) Is there any other solution which I've missed? How long does it take to run your check? If it takes less than 300 sec and is well within the limits of Lambda ( AWS Lambda Limits ), then schedule tasks with Lambda: Schedule Expressions Using Rate or Cron

navicat使用ssh登录mysql报错:expected key exchange group packet from server

白昼怎懂夜的黑 提交于 2019-12-03 23:52:10
一、navicat ssh连接方式 由于公司使用的是阿里云的RDS,需要通过内网连接数据库(不直接提供公网访问),此时可以使用navicat的ssh方式,通过一台跳板机连接内网的RDS 二、连接报错及解决方式 报错: SSH: expected key exchange group packet from server 解决方式: 1、升级navicat 2、修改sshd的配置 01 02 [root@Manager ~] #vim /etc/ssh/sshd_config #编辑sshd配置文件,在最后添加 KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1 参考文档: https://blog.csdn.net/enweitech/article/details/80677374 http://www.bubuko.com/infodetail-2993078.html 来源: https://www.cnblogs.com/grimm/p/11811944.html

唠唠 RDS 那些事 ——发布虚拟机

对着背影说爱祢 提交于 2019-12-03 20:39:45
引子 在上一篇中介绍了基于 Windows Server 2016 的RDS服务部署,大家有条件有环境的可以自己动手搭建一遍试试。如果你已经搭建好了环境,本篇将介绍的是如何创建集合,并把虚拟机发布出去供用户访问。 RDS 的集合种类 从大的方面来说,RDS 有两类集合:虚机集合和会话集合。虚机集合中又有池化虚机和个人虚机之分。我们的项目主要是提供个人虚机给用户,也就是每一个用户自己独占一台虚机的模式,因此本文主要针对此种模式进行介绍。 大家注意这两种模式在功能上的区别。 创建 RDS 集合 打开服务器管理器,导航到“远程桌面服务 概述”,点击“创建虚拟机集合”链接,开启创建集合向导。 创建集合向导的第一页是个提示页面,很多朋友不注意仔细阅读这个页面,可能认为 Windows 不就是下一步接着下一步吗。其实 Windows 远不是下一步这么简单的,而且个人认为微软在软件安装和服务设置的过程中的各种提示信息,做得还是挺不错的,仔细阅读这些提示可以避免走弯路。 接下来要给集合起个可以辨识的名称,全网这个名称是唯一的,另外为了更好地管理,也请在注释处稍加一些文字说明,以免时间长了、更换了管理人员后无法明了集合的作用。 接下来是选择集合类型,因本项目是要提供一人一机的服务,因此我们创建的是个人虚拟机集合。这类虚拟机,用户对它有完全控制权限。当然,这些权限也可以通过组策略来加以限定

navicat使用ssh登录mysql报错:expected key exchange group packet from server

孤街醉人 提交于 2019-12-03 15:17:24
一、navicat ssh连接方式 由于公司使用的是阿里云的RDS,需要通过内网连接数据库(不直接提供公网访问),此时可以使用navicat的ssh方式,通过一台跳板机连接内网的RDS 二、连接报错及解决方式 报错: SSH: expected key exchange group packet from server 解决方式: 1、升级navicat 2、修改sshd的配置 [root@Manager ~]#vim /etc/ssh/sshd_config #编辑sshd配置文件,在最后添加 KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1 来源: https://www.cnblogs.com/hujinzhong/p/11801921.html

Error: connect ETIMEDOUT rds lambda

匿名 (未验证) 提交于 2019-12-03 08:41:19
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I am trying to connect to RDS using Lambda function, but I am getting an error: var mysql = require('mysql'); exports.handler = function(event, context) { //Connect to RDS var connection = mysql.createConnection({ host : 'hostname', user : 'username', password : 'password', database : 'database' }); connection.connect( function(err) { if (err) { throw err; } else { console.log('DB connection establish'); } }); }; The error I am getting is: START RequestId: 9711e650-e582-11e5-af5f-97ba391a42ae Version: $LATEST 2016-03-08T23:08:06.737Z

RDS with Cloud Formation and AZ issues

匿名 (未验证) 提交于 2019-12-03 08:35:02
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I am using cloud formation to create a setup containing an RDS instance. I am having some difficulties creating the RDS Instance on the account of the following error: DB Subnet Group doesn't meet availability zone coverage requirement. Please add subnets to cover at least 2 availability zones. Current coverage: 1 The problem is that the entire setup is on a single AZ... what am i supposed to do? just create an extra subnet in a different AZ that has nothing in it just for the RDS? Maybe there is some way AWS can create that subnet

MySQL/Amazon RDS error: “you do not have SUPER priviledges…”

匿名 (未验证) 提交于 2019-12-03 02:16:02
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I'm attempting to copy my mysql database from an Amazon EC2 to an RDS: I successfuly did a mysqldump of my database into my root folder using this: root@ip-xx-xx-xx-xx:~# mysqldump my_database -u my_username -p > my_database.sql Then I tried to transfer this .sql file to my new RDS database: root@ip-xx-xx-xx-xx:~# mysql my_database -u my_username -p -h my_new_database.xxxxxxxxx.us-east-1.rds.amazonaws.com < my_database.sql Unfortunately, I get following error message: You do not have the SUPER privilege and binary logging is enabled (you

Connecting to Amazon RDS instance through EC2 instance using MySQL Workbench

匿名 (未验证) 提交于 2019-12-03 01:55:01
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 由 翻译 强力驱动 问题: In AWS I have a VPC set up with a Bastion Host. The bastion host is a single EC2 instance with a public address trough which you can SSH to any other server on the VPC. I have created an RDS MySQL instance within the VPC and I would like to connect to it using MySQL workbench. I have followed the steps detailed here , however in "Step 6: Setting up remote SSH Configuration", it asks me to "Provide the Public DNS of the Amazon EC2 instance" (i.e. the bastion host). MySQL workbench then does checks for certain MySQL resources on that

AWS RDS and pgAdmin4 : Unable to connect to server: could not connect to server: Connection timed out

匿名 (未验证) 提交于 2019-12-03 01:40:02
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I am trying to manipulate db directly from pgAdmin4 but I cannot connect. What I checked and did are I read this doc and following https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToPostgreSQLInstance.html#USER_ConnectToPostgreSQLInstance.Troubleshooting and input the information about the db instance as the doc does. However I couldn't connect and I checked the security group. The VPC security group is like this. What else should I check? I totally have no idea how I can fix this. The only concern is the password's current