Redshift. How can we transpose (dynamically) a table from columns to rows?

问题

How can we transpose a Redshift table from columns to rows?

For example, if we have a generic (not already known) table like the following:

source table:

date        id      alfa                beta                gamma   ...                 omega
2018-08-03  1       1                   2                   3                           4
2018-08-03  2       4                   3                   2                           1
...
2018-09-04  1       3                   1                   2                           4
...

How we can achieve the following result?

transposed table:

date        id      column_name     column_value
2018-08-03  1       alfa            1
2018-08-03  1       beta            2
...
2018-08-03  2       omega           1
...
2018-09-04  1       gamma           2
...

Where the target table, the number of columns (alfa, beta, gamma, ..., omega) are all dynamic (so We're looking for a solution that no case when mapping for each column is needed, since We'd like to apply this to several different tables).

But we will have and date and id fields in all target tables (or at last a primary key or a candidate key in all tables).

Our Redshift version is:

PostgreSQL 8.0.2, Redshift 1.0.3380

How can we do that?

回答1:

You would need to hard-code the column names into the query.

CREATE TABLE stack(date TEXT, id BIGINT, alpha INT, beta INT, gamma INT, omega INT);

INSERT INTO STACK VALUES('2018-08-03', 1, 1, 2, 3, 4);
INSERT INTO STACK VALUES('2018-08-03', 2, 4, 3, 2, 1);
INSERT INTO STACK VALUES('2018-08-04', 1, 3, 1, 2, 4);

SELECT
  date,
  id,
  col,
  col_value
FROM
(
SELECT date, id, alpha AS col_value, 'alpha' AS col FROM stack
UNION
SELECT date, id, beta  AS col_value, 'beta'  AS col FROM stack
UNION
SELECT date, id, gamma AS col_value, 'gamma' AS col FROM stack
UNION
SELECT date, id, omega AS col_value, 'omega' AS col FROM stack
) AS data
ORDER BY date, id, col

Results in:

2018-08-03  1   alpha   1
2018-08-03  1   beta    2
2018-08-03  1   gamma   3
2018-08-03  1   omega   4
2018-08-03  2   alpha   4
2018-08-03  2   beta    3
2018-08-03  2   gamma   2
2018-08-03  2   omega   1
2018-08-04  1   alpha   3
2018-08-04  1   beta    1
2018-08-04  1   gamma   2
2018-08-04  1   omega   4

回答2:

In lieu of not providing the answer in comments, here is the semi pseudo code to explain how I did it, let me know if you need more info/clarification

# dictionary to define your target structure
target_d = {'date':'','id':'','column_name':'','column_value':''}

# dictionary for source structure
source_d = {'date':'date','id':'id','column_name1':'','column_name2':''....}

with this dict above you are declaring if a field is mapped it will not be dynamic, all other fields/columns will be pivoted, you can enhance it to be dynamic using your source table DDL

# assuming you already read your source data
# your while loop to go thru the coming data
while <your code here>
    # create a dict to process an incoming row
    curr_d = target_d.copy()

    curr_d['date'] = date from incoming record
    curr_d['id'] = id from incoming record

    # since we are going to create a row for each column name/value combos 
    # we need a new dict to hold the values

    out_d = curr_d

this line above serves two purposes, create a new dict for output row and keep persistent part of the output row (i.e. date and id)

    # rest of the fields are going to be pivoted now
    for afield in source_d:
        if afield not in source_d.values():
            curr_d['column_name'] = afield
            curr_d['column_value'] = column value from incoming record

        create a 'row' from your out_d dict
        write to output/ append to output data frame (if you use a data frame)

While loop will go thru the your source rows, for loop will create a new row for each column name/value combo for target

Let me know if this works for you.

来源：https://stackoverflow.com/questions/52174526/redshift-how-can-we-transpose-dynamically-a-table-from-columns-to-rows

标签

dynamic

pivot-table

multiple-columns

amazon-redshift

transpose