amazon-redshift

问题 Im trying to load a CSV file while loading I got the error Delimited value missing end quote CSV file Error line REDSHIFT ERROR: line_number | 13 colname | Comment col_length | 250 raw_line |"123"|"123"|"xyz"|"2009-06-25 21:00:14.660000000"|"0"|""|""|""|""|""|"HI, raw_field_value | HI, err_code | 1214 err_reason | Delimited value missing end quote 回答1: You'll need to escape the newline characters in the source data and then specify the ESCAPE option in your COPY statement. Please have a look

问题 I am trying to setup a sync between AWS Aurora and Redshift. What is the best way to achieve this sync? Possible ways to sync can be: - Query table to find changes in a table(since I am only doing inserts, updates don't matter), export these changes to a flat file in S3 bucket and use Redshift copy command to insert into Redshift. Use python publisher and Boto3 to publish changes into a Kinesis stream and then consume this stream in Firehose from where I can copy directly into Redshift. Use

问题 I am having this problem when I am trying to copy to AWS redshift. This is the code I am trying to run: with open('path/to/files, 'rb') as fo: cursor.copy_from(fo, 'schema.table', sep=',') cursor.commit() And I encountered the error: psycopg2.ProgrammingError: syntax error at or near "stdin" LINE 1: ...Y schema.table FROM stdin WITH... I am running python 3.5 with psycopg2. Hope that you guys can help! Thx in advance! 回答1: AWS Redshift is not PostgreSQL, though it supports a subset of

问题 My team has views built upon views built upon views, so often a DROP TABLE CASCADE is a recipe for disaster and a lot of trial and error. What I would like is a query that returns all dependent objects that need to be re-created in the correct order given a certain schema and table so that they could be automated and ran in a script. I'm working with a modified version of the dependency query on the Redshift DROP TABLE documentation http://docs.aws.amazon.com/redshift/latest/dg/r_DROP_TABLE

问题 I have a field in a redshift column that looks like the following: abcd1234df-TEXT_I-WANT the characters and numbers in the first 10 digits can be either letters or numbers. If I use a capture group regex, I would use a poorly written expression like (\w\w\w\w\w\w\w\w\w\w\W)(.*) and grap the 2nd group But I'm having trouble implementing this in redshift, so not sure how I can grab only the stuff after the first hyphen 回答1: As mentioned before, regex might be an overkill. However, it could be

问题 I have a field in a redshift column that looks like the following: abcd1234df-TEXT_I-WANT the characters and numbers in the first 10 digits can be either letters or numbers. If I use a capture group regex, I would use a poorly written expression like (\w\w\w\w\w\w\w\w\w\w\W)(.*) and grap the 2nd group But I'm having trouble implementing this in redshift, so not sure how I can grab only the stuff after the first hyphen 回答1: As mentioned before, regex might be an overkill. However, it could be

问题 The following works in Postgres 9.6 but not in Redshift: ALTER TABLE stats ADD COLUMN IF NOT EXISTS panel_exit timestamp; Can the same functionality be achieved in Redshift? 回答1: There is no Amazon Redshift command equivalent to ADD COLUMN IF NOT EXISTS . See: ALTER TABLE documentation To replicate this functionality, your application would first need to query the table metadata and then make the decision whether to issue the ADD COLUMN command. 回答2: John's answer set me in the right

问题 When I unload a table from amazon redshift to S3, it always splits the table into two parts no matter how small the table. I have read the redshift documentation regarding unloading, but no answers other than it says sometimes it splits the table (I've never seen it not do that). I have two questions: Has anybody every seen a case where only one file is created? Is there a way to force redshift to unload into a single file? 回答1: Amazon recently added support for unloading to a single file by

问题 When I unload a table from amazon redshift to S3, it always splits the table into two parts no matter how small the table. I have read the redshift documentation regarding unloading, but no answers other than it says sometimes it splits the table (I've never seen it not do that). I have two questions: Has anybody every seen a case where only one file is created? Is there a way to force redshift to unload into a single file? 回答1: Amazon recently added support for unloading to a single file by

问题 Is it possible to fire a copy command from S3 To Redshift through java jdbc connection? Example: copy test from 's3://' CREDENTIALS 'aws_access_key_id=xxxxxxx;aws_secret_access_key=xxxxxxxxx' 回答1: Yes try code as below String dbURL = "jdbc:postgresql://x.y.us-east-1.redshift.amazonaws.com:5439/dev"; String MasterUsername = "userame"; String MasterUserPassword = "password"; Connection conn = null; Statement stmt = null; try{ //Dynamically load postgresql driver at runtime. Class.forName("org