amazon-redshift

Export JSON data and load into a Relational Database

久未见 提交于 2019-12-12 16:24:29
问题 Is there a standard way of converting JSON data to relational database with multiple tables? We need to export our data from a MongoDB instance and import them into a Redshift cluster. The only problem is that some of MongoDB fields contain objects and arrays. Redshift cluster accepts CSV so I'm thinking the output would be at least one CSV file for each new table. I don't need specific implementation. I just wanted to know the concepts on how to efficiently transform the JSON/NoSQL data into

Defining Tables, Views, and Indexes in AWS CloudFormation

做~自己de王妃 提交于 2019-12-12 13:13:28
问题 In AWS CloudFormation, you can define a stack of services with a JSON/YAML template. In particular, I am using Redshift and Aurora clusters. The template allows you to define administrative properties of the cluster, but there doesn't seem to be a clear way to define tables, views, and indexes in the cluster such that when I push updates of stack, the DB's automatically update the schema as needed, similar to the way it manages updates to full stacks. Is there a way to manage the schema of

AWS RedShift - .NET Core (ODBC Support?)

自古美人都是妖i 提交于 2019-12-12 12:13:28
问题 How can I connect and run queries against AWS RedShift using .NET Core? Code sample please. I have gone through the AWS docs and .Net Core docs but no luck. 回答1: This answer is one for a particular point in time and won't age well... The EntityFramework Core project is the one I'd keep the closest eye on. The lack of ODBC is well known, especially for those who want to connect to Oracle. For the time being, you may need to fork an Oracle client for .NET core and make modifications as

Conditionally drop temporary table in Redshift

无人久伴 提交于 2019-12-12 12:12:10
问题 We are using http://aws.amazon.com/redshift/ and I am creating/dropping temporary tables in reports. Occasionally we encounter cases where someone has created a temporary table and failed to drop it. In other databases, for instance PostgreSQL which Redshift is based on, I could simply: DROP TEMP TABLE IF EXISTS tblfoo; But that is a syntax error in Redshift. I can check for the existence of temporary tables myself using http://docs.aws.amazon.com/redshift/latest/dg/r_STV_TBL_PERM.html but

Loading CSV data with NaN into AWS Redshift

元气小坏坏 提交于 2019-12-12 10:39:34
问题 I am trying to load a CSV file from AWS S3 into AWS Redshift. The CSV file contains a line like: 15,NaN,0 The table was created via: CREATE TABLE foo (a INT, b DOUBLE PRECISION, c INT); And I am trying to load the table using the COPY command: COPY foo (a, b, c) FROM "s3://" CREDENTIALS ... CSV; And I get an error complaining: Invalid digit value 'N' Trying to load that same line via an INSERT statement manually works just fine: INSERT INTO foo (a, b, c) VALUES (15, 'NaN', 0); Any help would

NpgsqlConnection hangs for hours on opening connection

感情迁移 提交于 2019-12-12 10:18:47
问题 I was able to successfully connect using Npgsql nuget package, until one day the program started hanging for hours on executing conn.Open() without returning. I can successfully connect with Aginity client with same parameters. I tried Firewall On and Off. I tried Antivirus On and Off. I tried Fiddler On and Off. Some things to Note: I need to be on VPN to connect to that host. It just hangs for hours, when conn.Open() is executed. When I am not on VPN, it times out immediately. We can't see

How to find the ODBC driver name for a connection string?

你离开我真会死。 提交于 2019-12-12 09:53:43
问题 Whenever I use ODBC drivers with a full connection string, and not just a DSN entry, I often get an error similar to this Data source name not found and no default driver specified I have the correct syntax of the connection string (or so the Internet says), but I suspect I don't know the correct name for the current version of the ODBC driver I am using How do I find the correct name, for either 32 or 64 bit? 回答1: Use the ODBC Data Source Administrator app. Make sure you use the 32 bit or

How can I copy an IDENTITY field?

梦想的初衷 提交于 2019-12-12 08:41:59
问题 I’d like to update some parameters for a table, such as the dist and sort key. In order to do so, I’ve renamed the old version of the table, and recreated the table with the new parameters (these can not be changed once a table has been created). I need to preserve the id field from the old table, which is an IDENTITY field. If I try the following query however, I get an error: insert into edw.my_table_new select * from edw.my_table_old; ERROR: cannot set an identity column to a value [SQL

Redshift data access from node jdbc

筅森魡賤 提交于 2019-12-12 06:14:21
问题 I am getting below error when I try to data from redshift with the below mentioned code var jdbc = new ( require('jdbc') ); var config = { libpath: 'C:/Users/ABCD/Desktop/jar/RedshiftJDBC41-1.1.6.1006.jar', //libs: [__dirname + 'path/to/other/jars.jar'], drivername: 'com.amazon.redshift.jdbc41.Driver', url: 'jdbc:redshift://examplecluster.abc123xyz789.us-west-2.redshift.amazonaws.com:5439/dev', user: 'xxxx', password: 'xxxxx' }; jdbc.initialize(config, function(err, res) { if (err) { console