postgresql

Export query results from BigQuery to Postgres

血红的双手。 提交于 2021-02-07 19:59:16
问题 I am trying to export the results of a query in BigQuery, and get the data into Postgres. The data may be as much 250 million record, ~26Gb. Option 1: Save query results to a temp table Export table to csv(s) Bulk upsert to postgres (This will be slow) Option 2: Somehow get the two DBs to speak directly I don't know if this is possible Thank you for any information!!! 回答1: This BigQuery Foreign Data Wrapper for postgreSQL allows you to query BigQuery directly from within PostgreSQL. Using

RANGE PRECEDING is only supported with UNBOUNDED

吃可爱长大的小学妹 提交于 2021-02-07 19:57:10
问题 I want to try avg() aggregation within a time window sql code select user_id,timestamp avg(y) over(range between '5 second' preceding and '5 second' following), from A but the system report error RANGE PRECEDING is only supported with UNBOUNDED Is there any method to implement, say, a 10 second window for avg() window function? frame of window function is as wide as range from n seconds preceding the timestamp of current row and m seconds following the timestamp of current row 回答1: RANGE

How to import ZIpped file into Postgres Table

我怕爱的太早我们不能终老 提交于 2021-02-07 19:28:03
问题 I would like to important a file into my Postgresql system(specificly RedShift). I have found a arguement for copy that allows importing a gzip file. But the provider for the data I am trying to include in my system only produces the data in a .zip. Any built in postgres commands for opening a .zip? 回答1: From within Postgres: COPY table_name FROM PROGRAM 'unzip -p input.csv.zip' DELIMITER ','; From the man page for unzip -p : -p extract files to pipe (stdout). Nothing but the file data is

PostgreSQL partial unique index and upsert

被刻印的时光 ゝ 提交于 2021-02-07 19:11:51
问题 I'm trying to do an upsert to a table that has partial unique indexes create table test ( p text not null, q text, r text, txt text, unique(p,q,r) ); create unique index test_p_idx on test(p) where q is null and r is null; create unique index test_pq_idx on test(p, q) where r IS NULL; create unique index test_pr_idx on test(p, r) where q is NULL; In plain terms, p is not null and only one of q or r can be null. Duplicate inserts throw constraint violations as expected insert into test(p,q,r

PostgreSQL partial unique index and upsert

六眼飞鱼酱① 提交于 2021-02-07 19:10:03
问题 I'm trying to do an upsert to a table that has partial unique indexes create table test ( p text not null, q text, r text, txt text, unique(p,q,r) ); create unique index test_p_idx on test(p) where q is null and r is null; create unique index test_pq_idx on test(p, q) where r IS NULL; create unique index test_pr_idx on test(p, r) where q is NULL; In plain terms, p is not null and only one of q or r can be null. Duplicate inserts throw constraint violations as expected insert into test(p,q,r

Decimal value in Postgresql returned as String in Node.js

回眸只為那壹抹淺笑 提交于 2021-02-07 19:09:22
问题 When I run a query to my postgresql database on a node.js server, the value that I am getting is of variable type string when in fact it is a decimal in the postgresql database. I am not sure why my decimals or even bigInts are returning as type strings. I am using knex as my ORM if that makes a difference. So far what I have read online is not very clear about what to do, and it seems as if this happens automatically to preserve precision??? What is the best work-around for this? Is it best

How do I use variables in a postgresql function for loop query

半世苍凉 提交于 2021-02-07 18:32:06
问题 I have a rather complicated function in postgresql (Version 9.4.4) that I need a bit of help with. I have a loop (with lots of work below) declared like this inside of my function: CREATE OR REPLACE function getRSI( psymbol varchar, pstarttime timestamp with time zone, pendtime timestamp with time zone, pduration double precision, ptable varchar ) RETURNS SETOF rsi AS $BODY$ declare row_data record; -- some variables begin FOR row_data IN SELECT datetime, value FROM "4" WHERE symbol = 'AAPL'

How do I use variables in a postgresql function for loop query

拈花ヽ惹草 提交于 2021-02-07 18:32:00
问题 I have a rather complicated function in postgresql (Version 9.4.4) that I need a bit of help with. I have a loop (with lots of work below) declared like this inside of my function: CREATE OR REPLACE function getRSI( psymbol varchar, pstarttime timestamp with time zone, pendtime timestamp with time zone, pduration double precision, ptable varchar ) RETURNS SETOF rsi AS $BODY$ declare row_data record; -- some variables begin FOR row_data IN SELECT datetime, value FROM "4" WHERE symbol = 'AAPL'

How to create a trigger function dynamically in pgsql?

我怕爱的太早我们不能终老 提交于 2021-02-07 15:00:10
问题 I want to write a pgsql function to create trigger dynamically. For example, a trigger to count insertions in each table. I've tried EXECUTE like this: CREATE FUNCTION trigen(tbl text) RETURNS void AS $$ BEGIN EXECUTE format( 'CREATE FUNCTION %s_insertCnt() RETURNS TRIGGER AS $$ BEGIN UPDATE insertions SET n = n + 1 WHERE tablename = %s; END $$ LANGUAGE plpgsql', tbl, quote_nullable(tbl)); EXECUTE format('CREATE TRIGGER %s_inCnt BEFORE INSERT ON %s FOR EACH ROW EXECUTE PROCEDURE %s_insertCnt(

ProgrammingError: can't adapt type 'set'

旧巷老猫 提交于 2021-02-07 14:53:21
问题 I am importing Excel data into postgreSQL using Python and running into an Programming Error. I did research the issue and found out it has something to do with postgreSQL. Can someone please provide assistance. import psycopg2 import xlrd book = xlrd.open_workbook("T:\data.xlsx") sheet = book.sheet_by_name("HCSData") database = psycopg2.connect (database = "", user="") cursor = database.cursor() delete = """Drop table if exists "Python".hcsdata""" print (delete) mydata = cursor.execute