amazon-redshift

How to connect Amazon Redshift to python

 ̄綄美尐妖づ 提交于 2020-08-21 06:11:26
问题 This is my python code and I want to connect my Amazon Redshift database to Python, but it is showing error in host. Can anyone tell me the correct syntax? Am I passing all the parameters correctly? con=psycopg2.connect("dbname = pg_table_def, host=redshifttest-icp.cooqucvshoum.us-west-2.redshift.amazonaws.com, port= 5439, user=me, password= secret") This is the error: OperationalError: could not translate host name "redshift://redshifttest-xyz.cooqucvshoum.us-west-2.redshift.amazonaws.com,"

How to disable using cache results in Redshift Query?

天大地大妈咪最大 提交于 2020-08-19 20:34:05
问题 I am interested in performance testing my query in Redshift. I would like to disable the query from using any cached results from prior queries. In other words, I would like the query to run from scratch. Is it possible to disable cached results only for the execution of my query? I would not like to disable cached results for the entire database/all queries. 回答1: SET enable_result_cache_for_session TO OFF; From enable_result_cache_for_session - Amazon Redshift: Specifies whether to use query

Quote escaped quotes in Redshift external tables

旧巷老猫 提交于 2020-07-21 06:14:12
问题 I'm trying to create an external table in Redshift from a csv that has quote escaped quotes in it, as documented in rfc4180: If double-quotes are used to enclose fields, then a double-quote appearing inside a field must be escaped by preceding it with another double quote. For example: "aaa","b""bb","ccc" I get no errors but the final table has a null value where my string should be. Is there a way to tell Redshift to understand this csv format when creating an external table? I do not want

Is there a way to do a SQL dump from Amazon Redshift

蓝咒 提交于 2020-07-17 06:26:08
问题 Is there a way to do a SQL dump from Amazon Redshift? Could you use the SQL workbench/J client? 回答1: We are currently using Workbench/J successfuly with Redshift. Regarding dumps, at the time there is no schema export tool available in Redshift (pg_dump doesn't work), although data can always be extracted via queries. Hope to help. EDIT: Remember that things like sort and distribution keys are not reflected on the code generated by Workbench/J. Take a look to the system table pg_table_def to

Copying data from S3 to Redshift hangs

久未见 提交于 2020-07-05 12:56:28
问题 I've been trying to load data into Redshift for the last couple of days with no success. I have provided the correct IAM role to the cluster, I have given access to S3 , I am using the COPY command with either the AWS credentials or the IAM role and so far no success. What can be the reason for this? It has come to the point that I don't have many options left. So the code is pretty basic, nothing fancy there. See below: copy test_schema.test from 's3://company.test/tmp/append.csv.gz' iam

Avoid nested aggregate error using coalesce()

浪子不回头ぞ 提交于 2020-07-03 13:41:27
问题 I currently have a query using coalesce that worked in SQL server,however, it is not working in Amazon Redshift. Is there a way I can more appropriately write this to use in Redshift: coalesce(sum(Score)/nullif(sum(ScorePrem),0),0) as percent 回答1: Consider running the aggregate query as a subquery or CTE, then handle transformation or secondary calculations in an outer main query. WITH agg AS ( SELECT calendar_month_id ,day_of_month ,month_name ,DaysRemaining ,RPTBRANCH ,0 AS TotalGrp ,SUM

inserting multiple records at once into Redshift with R

心已入冬 提交于 2020-06-26 05:19:23
问题 I need to push a few thousand rows into Redshift multiple times a day. But I can't use bulk Insert from S3 for administrated reasons. What's the fastest way? Details: There are 3 ways (that I can see) to insert rows into a table in Amazon Redshift from R: Row by row insert query. Each row is inserted as its own INSERT VALUES query Multiple Row insert query: same as 1, but multiple records are inserted per query. The size is constrained either by number of rows or the 16MB max SQL query size

Trying to count cumulative distinct entities using Redshift SQL

China☆狼群 提交于 2020-06-24 13:46:46
问题 I'm trying to get a cumulative count of distinct objects in Redshift over a time series. The straightforward thing would be to use COUNT(DISTINCT myfield) OVER (ORDER BY timefield DESC ROWS UNBOUNDED PRECEDING), but Redshift gives a "Window definition is not supported" error. For example, the code below is trying to find the cumulative distinct users for every week from the first week to the present. However, I get the "Window function not supported" error. SELECT user_time.weeks_ago, COUNT