PostgreSQL Primary Key sequence lost after migration using AWS DMS

问题

I recently migrated a self hosted Postgres database to AWS RDS using the AWS Migration Service.
Postgres Version: 10.6

I have noticed that all of my primary keys are no longer set to "sequence", and when I attempt to manually add a sequence it starts at 1 instead of continuing the count that is already set.

I use Rails with the database, so my sql skills are pretty low. I can generally find my way around inserts and updates, but this is not a realm I have much experience in.

My question has 2 parts:

How do I fix a single table? I want to understand and test what I am doing before moving on.
Is there a way I can apply this fix to every table I have without needing to manually modify each table?

回答1:

After @a_horse_with_no_name pointed me in the right direction and chatting with AWS I am able to answer my own question, at least if you are using AWS Database Migration Service (DMS).

The problem is, DMS only focuses on the data itself and not really the schema (which to me seems like a major oversight, especially if your using the same database technology but that is another issue). So the schema itself is not migrated. The documentation does not really make this clear.

To fix this issue:

Stop (if it still exists) the existing AWS DMS migration
Drop the existing migrated database, and create a new empty schema to use
Follow the steps here https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Installing.html to install and setup the Amazon Schema Conversation Tool (SCT)
Once you are connected to both databases, follow the steps here https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Converting.html to "convert" your schema (I did the entire "public" schema for this database to ensure everything is covered
Create or modify your AWS DMS Migration, ensuring Target Table Preparation Mode = "TRUNCATE" and disable foreign keys on the target database. If modifying, make sure when asked you "RESTART" not resume

What I have not yet tested is how to handle the fact that I am migrating a live database. So the sequences may be out of date on the target database when the migration is done. I believe I can just later go into SCT and only migrate the sequences but I have not tested this yet.

回答2:

Unless you missed something and the migration could be processed once more (with different parameters? I have never used AWS Migration Service but I presume it should preserve the serial-iness of your columns...) you'll need to re-create the sequences too.

I encountered a similar situation a year or so ago, and wrote this answer on SO.

Here's the gist of it:

CREATE SEQUENCE foo_a_seq OWNED BY foo.a;
SELECT setval('foo_a_seq', coalesce(max(a), 0)) FROM foo;
ALTER TABLE foo ALTER COLUMN a SET DEFAULT nextval('foo_a_seq');

This would create a sequence foo_a_seq, with its nextval being one higher than the max value in foo.a (or 1 if there is no foo record).

I also went ahead and set up a Function to quickly apply to all the tables/columns in need:

CREATE OR REPLACE FUNCTION make_into_serial(table_name TEXT, column_name TEXT) RETURNS INTEGER AS $$
DECLARE
    start_with INTEGER;
    sequence_name TEXT;
BEGIN
    sequence_name := table_name || '_' || column_name || '_seq';
    EXECUTE 'SELECT coalesce(max(' || column_name || '), 0) + 1 FROM ' || table_name
            INTO start_with;
    EXECUTE 'CREATE SEQUENCE ' || sequence_name ||
            ' START WITH ' || start_with ||
            ' OWNED BY ' || table_name || '.' || column_name;
    EXECUTE 'ALTER TABLE ' || table_name || ' ALTER COLUMN ' || column_name ||
            ' SET DEFAULT nextVal(''' || sequence_name || ''')';
    RETURN start_with;
END;
$$ LANGUAGE plpgsql VOLATILE;

Use it like so:

SELECT make_into_serial('foo', 'a');

来源：https://stackoverflow.com/questions/55633029/postgresql-primary-key-sequence-lost-after-migration-using-aws-dms

标签

postgresql

amazon-rds