Loading initial data with Django 1.7 and data migrations

Question

I recently switched from Django 1.6 to 1.7, and I began using migrations (I never used South).

Before 1.7, I used to load initial data with a fixture/initial_d

Answer 1

On Django 2.1, I wanted to load some models (Like country names for example) with initial data.

But I wanted this to happen automatically right after the execution of initial migrations.

So I thought that it would be great to have an sql/ folder inside each application that required initial data to be loaded.

Then within that sql/ folder I would have .sql files with the required DMLs to load the initial data into the corresponding models, for example:

INSERT INTO appName_modelName(fieldName)
VALUES
    ("country 1"),
    ("country 2"),
    ("country 3"),
    ("country 4");

To be more descriptive, this is how an app containing an sql/ folder would look:

Also I found some cases where I needed the sql scripts to be executed in a specific order. So I decided to prefix the file names with a consecutive number as seen in the image above.

Then I needed a way to load any SQLs available inside any application folder automatically by doing python manage.py migrate.

So I created another application named initial_data_migrations and then I added this app to the list of INSTALLED_APPS in settings.py file. Then I created a migrations folder inside and added a file called run_sql_scripts.py (Which actually is a custom migration). As seen in the image below:

I created run_sql_scripts.py so that it takes care of running all sql scripts available within each application. This one is then fired when someone runs python manage.py migrate. This custom migration also adds the involved applications as dependencies, that way it attempts to run the sql statements only after the required applications have executed their 0001_initial.py migrations (We don't want to attempt running a SQL statement against a non-existent table).

Here is the source of that script:

import os
import itertools

from django.db import migrations
from YourDjangoProjectName.settings import BASE_DIR, INSTALLED_APPS

SQL_FOLDER = "/sql/"

APP_SQL_FOLDERS = [
    (os.path.join(BASE_DIR, app + SQL_FOLDER), app) for app in INSTALLED_APPS
    if os.path.isdir(os.path.join(BASE_DIR, app + SQL_FOLDER))
]

SQL_FILES = [
    sorted([path + file for file in os.listdir(path) if file.lower().endswith('.sql')])
    for path, app in APP_SQL_FOLDERS
]


def load_file(path):
    with open(path, 'r') as f:
        return f.read()


class Migration(migrations.Migration):

    dependencies = [
        (app, '__first__') for path, app in APP_SQL_FOLDERS
    ]

    operations = [
        migrations.RunSQL(load_file(f)) for f in list(itertools.chain.from_iterable(SQL_FILES))
    ]

I hope someone finds this helpful, it worked just fine for me!. If you have any questions please let me know.

NOTE: This might not be the best solution since I'm just getting started with django, however still wanted to share this "How-to" with you all since I didn't find much information while googling about this.