Mocha testing PostgreSQL with Knex is giving me a MigrationLocked error

Question

I have got a local development environment working for Node/PostgreSQL/Knex in the sense that I can post to a development database on my machine using an API. I am now trying to create tests for this functionality but am getting an error.

Here's my config:

//knexfile.js
module.exports = {
  development: {
    client: 'pg',
    connection: {
        host: '127.0.0.1',
        user: 'dbUser',
        password: 'dbpword',
        port: 5432,
        database: 'example-name'
    },
    migrations: {
      directory: __dirname + '/db/migrations'
    },
    seeds: {
      directory: __dirname + '/db/seeds/development'
    }
  },
}

//db.js
const config = require('../knexfile.js');
const env = process.env.NODE_ENV || 'development';
const knex = require("knex")(config[env]);

module.exports = knex;

knex.migrate.latest([config]);

And then my tests:

import chai from 'chai';
import { expect } from 'chai';
import chaiHttp from 'chai-http';
import knex from '../../db/db';
import app from '../../server';

chai.use(chaiHttp);

describe('Tokens API', () => {

  beforeEach((done) => {
   knex.migrate.rollback()
   .then(() => {
     knex.migrate.latest()
     .then(() => {
       return knex.seed.run()
       .then(() => {
        done();
       });
     });
   });
});

afterEach((done) => {
  knex.migrate.rollback()
  .then(() => {
    done();
  });
});

describe('POST /users', () => {
  it('posts a list of users to the database with all mandatory fields', (done) => {
    chai.request(app)
    .post('/users')
    .send({
        "users": [
              "steve",
              "whoever",
              "matt",
              "another"]})
      .end((err, res) => {
        expect(err).to.be.null;
        expect(res).to.have.status(200);
        expect(res).to.be.json;
        done();
      });
    });
  });
});

When I run this I get the following error twice - I think for the knex calls in the beforeEach block:

Knex:warning - Can't take lock to run migrations: Migration table is already locked
Knex:warning - If you are sure migrations are not running you can release the lock manually by deleting all the rows from migrations lock table: knex_migrations_lock
Unhandled rejection MigrationLocked: Migration table is already locked

I have tried numerous things - including clearing out the knex_migrations_lock table. The only support I can find online is this thread, which suggests clearing out the lock table using DELETE FROM Migrations_lock where id <> 0;, however my lock table only has a is_locked column with zero values.

Any idea what's going on?

EDIT: I've just realised if you edit out all the knex calls, the test actually passes. Could this be because I am effectively calling knex twice - once from db.js and once indirectly through server.js? If that's the case, how do I avoid doing this - because surely I need to call the knex setup for Node to run it?

Answer 1

For anyone stumbling across this, the problem was actually coming from db.js, specifically the last line:

const config = require('../knexfile.js');
const env = process.env.NODE_ENV || 'development';
const knex = require("knex")(config[env]);

module.exports = knex;

knex.migrate.latest([config]);

Of course this is asynchronous, and the tests were importing this file before trying to run their own knex functions, causing the lock. I got round this by adding a clause to block this running while testing:

if(process.env.NODE_ENV != 'test') {
   knex.migrate.latest([config])
}

You can then create a test environment by adding process.env.NODE_ENV='test' to each spec file, or by installing the npm env test module.

Answer 2

Had the exact same issue, ended up being due to my API calling the database when being initialized by the supertest library.

For example, my test file:

var db = require('../db');
var api = require('../api');

var supertest = require('supertest')(api);

describe('Session routes', () => {
  beforeEach((done) => {
    db.migrate.rollback()
      .then(() => {
        db.migrate.latest()
          .then(() => {
            return db.seed.run()
              .then(() => {
                done();
              });
          });
      });
  });

  afterEach((done) => {
    db.migrate.rollback()
      .then(() => {
        done();
      });
  });

  it('GET /session should error with no token', (done) => {
    supertest
      .get('/session')
      .set('Accept', 'application/json')
      .expect('Content-Type', /json/)
      .expect(401, {
        error: 'Unauthorized'
      }, done);
  });
});

On line 2, it requires my api - when my api is required the following code gets run straight away to initialize my api's external services API:

var db = require('./other-postgres-library');
var servicesApi = require('./services/api')(db);

This would connect to a bunch of external services and write the results to the database.

So when tests were running my APP was throwing errors because it was trying to write to a database which was being rolled back/migrated/seeded etc.

I changed my inner services API to initialize lazily and all my problems have disappeared.

In your case, I would hazard to guess when your tests runs this line import app from '../../server'; your app/server code is trying to running some queries against the database.