Shell script to execute pgsql commands in files

Question

I am trying to automate a set of procedures that create TEMPLATE databases.

I have a set of files (file1, file2, ... fileN), each of which contains a set of pgsql commands required for creating a TEMPLATE database.

The contents of the file (createdbtemplate1.sql) looks roughly like this:

CREATE DATABASE mytemplate1 WITH ENCODING 'UTF8';

\c mytemplate1

CREATE TABLE first_table ( 
 --- fields here ..
);

-- Add C language extension + functions
\i db_funcs.sql

I want to be able to write a shell script that will execute the commands in the file, so that I can write a script like this:

# run commands to create TEMPLATE db mytemplate1
# ./groksqlcommands.sh createdbtemplate1.sql

for dbname in foo foofoo foobar barbar
do
    # Need to simply create a database based on an existing template in this script
    psql CREATE DATABASE $dbname TEMPLATE mytemplate1
done

Any suggestions on how to do this? (As you may have guessed, I'm a shell scripting newbie.)

Edit

To clarify the question further, I want to know:

1. How to write groksqlcommands.sh (a bash script that will run a set of pgsql cmds from file)
2. How to create a database based on an existing template at the command line

Answer 1

First off, do not mix psql meta-commands and SQL commands. These are separate sets of commands. There are tricks to combine those (using the psql meta-commands \o and \\ and piping strings to psql in the shell), but stay away from that if you can.

• Make your files contain only SQL commands.
• Do not include the CREATE DATABASE statement in the SQL files. Create the db separately, you have multiple files you want to execute in the same template db.

I assume you are operating as system user postgres and you have postgres as Postgres superuser. All databases in the same db cluster on the default port 5432 and the user postgres has password-less access due to an IDENT setting in pg_hba.conf (default settings).

psql postgres -c "CREATE DATABASE mytemplate1 WITH ENCODING 'UTF8'
                  TEMPLATE template0"

I base the new template db on the system template db template0. Read more about that here.
Your question:

How to (...) run a set of pgsql cmds from file

Try:

psql mytemplate1 -f file

Example for batch:

#! /bin/sh

for file in /path/to/files/*; do
    psql mytemplate1 -f "$file"
done

The command option -f makes psql execute SQL commands in a file.

How to create a database based on an existing template at the command line

psql -c 'CREATE DATABASE myDB TEMPLATE mytemplate1'

The command option -c makes psql execute a single SQL command string. Can be multiple commands, terminated by ; - will be executed in one transaction and only the result of the last command returned.
Read about psql command options in the manual.

If you don't provide a database to connect to, psql will use the default maintenance database named postgres. In the second answer it is irrelevant which database we connect to.

Answer 2

you can echo your commands to the psql input:

for dbname in foo foofoo foobar barbar
do
    echo """
CREATE DATABASE $dbname TEMPLATE mytemplate1
""" | psql
done

Answer 3

If you're willing to go the extra mile, you'll probably have more success with sqlalchemy. It'll allow you to build scripts with python instead of bash, which is easier and has better control.

As requested in the comments: https://github.com/srathbun/sqlCmd

Answer 4

Store your sql scripts under a root dir Use dev,tst,prd parametrized dbs Use find to run all your pgsql scripts as shown here Exit on errors

Or just git clone the whole tool from here