Stored procedures/DB schema in source control

Question

Do you guys keep track of stored procedures and database schema in your source control system of choice?

When you make a change (add a table, update an stored proc,

Answer 1

In past experiences, I've kept database changes source controlled in such a way that for each release of the product any database changes were always scripted out and stored in the release that we're working on. The build process in place would automatically bring the database up to the current version based on a table in the database that stored the current version for each "application". A custom .net utility application we wrote would then run and determine the current version of the database, and run any new scripts against it in order of the prefix numbers of the scripts. Then we'd run unit tests to make sure everything was all good.

We'd store the scripts in source control as follows (folder structure below):

I'm a little rusty on current naming conventions on tables and stored procedures so bare with my example...

[root]
    [application]
        [version]
            [script]

\scripts
    MyApplication\
        1.2.1\
            001.MyTable.Create.sql
            002.MyOtherTable.Create.sql
            100.dbo.usp.MyTable.GetAllNewStuff.sql

With the use of a Versions table that would take into account the Application and Version the application would restore the weekly production backup, and run all the scripts needed against the database since the current version. By using .net we were easily able to package this into a transaction and if anything failed we would rollback, and send emails out, so we knew that release had bad scripts.

So, all developers would make sure to maintain this in source control so the coordinated release would make sure that all the scripts we plan to run against the database would run successfully.

This is probably more information than you were looking for, but it worked very well for us and given the structure it was easy to get all developers on board.

When release day came around the operations team would follow the release notes and pick up the scripts from source control and run the package against the database with the .net application we used during the nightly build process which would automatically package the scripts in transactions so if something failed it would automatically roll back and no impact to the database was made.

Answer 2

I agree with (and upvote) Robert Paulson's practice. That is assuming you are in control of a development team with the responsibility and discipline to adhere to such a practice.

To "force" that onto my teams, our solutions maintain at least one database project from Visual Studio Team Edition for Database Professionals. As with other projects in the solution, the database project gets versioned control. It makes it a natural development process to break the everything in the database into maintainable chunks, "disciplining" my team along the way.

Of course, being a Visual Studio project, it is no where near perfect. There are many quirks you will run into that may frustrate or confuse you. It takes a fair bit of understanding how the project works before getting it to accomplish your tasks. Examples include

• deploying data from CSV files.
• selective deployment of test data based on build type.
• Visual Studio crashing on comparing with databases with certain type of CLR assembly embedded within.
• no means of differntiation between test/production databases that implement different authentication schemes - SQL users vs Active Directory users.

But for teams who don't have a practice of versioning their database objects, this is a good start. The other famous alternative is of course, Red Gate's suite of SQL Server products, which most people who use them consider superior to Microsoft's offering.

Answer 3

The solution we used at my last job was to number the scripts as they were added to source control:

01.CreateUserTable.sql
02.PopulateUserTable
03.AlterUserTable.sql
04.CreateOrderTable.sql

The idea was that we always knew which order to run the scripts, and we could avoid having to manage data integrity issues that might arise if you tried modifying script #1 (which would presumable cause the INSERTs in #2 to fail).

Answer 4

For procs, write the procs with script wrappers into plain files, and apply the changes from those files. If it applied correctly, then you can check in that file, and you'll be able to reproduce it from that file as well.

For schema changes, you may need to check in scripts to incrementally make the changes you've made. Write the script, apply it, and then check it in. You can build a process then, to automatically apply each schema script in series.

Answer 5

We choose to script everything, and that includes all stored procedures and schema changes. No wysiwyg tools, and no fancy 'sync' programs are necessary.

Schema changes are easy, all you need to do is create and maintain a single file for that version, including all schema and data changes. This becomes your conversion script from version x to x+1. You can then run it against a production backup and integrate that into your 'daily build' to verify that it works without errors. Note it's important not to change or delete already written schema / data loading sql as you can end up breaking any sql written later.

-- change #1234
ALTER TABLE asdf ADD COLUMN MyNewID INT
GO

-- change #5678
ALTER TABLE asdf DROP COLUMN SomeOtherID
GO

For stored procedures, we elect for a single file per sproc, and it uses the drop/create form. All stored procedures are recreated at deployment. The downside is that if a change was done outside source control, the change is lost. At the same time, that's true for any code, but your DBA'a need to be aware of this. This really stops people outside the team mucking with your stored procedures, as their changes are lost in an upgrade.

Using Sql Server, the syntax looks like this:

if exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[usp_MyProc]') and OBJECTPROPERTY(id, N'IsProcedure') = 1)
drop procedure [usp_MyProc]
GO

CREATE PROCEDURE [usp_MyProc]
(
    @UserID INT
)
AS

SET NOCOUNT ON

-- stored procedure logic.

SET NOCOUNT OFF

GO  

The only thing left to do is write a utility program that collates all the individual files and creates a new file with the entire set of updates (as a single script). Do this by first adding the schema changes then recursing the directory structure and including all the stored procedure files.

As an upside to scripting everything, you'll become much better at reading and writing SQL. You can also make this entire process more elaborate, but this is the basic format of how to source-control all sql without any special software.

addendum: Rick is correct that you will lose permissions on stored procedures with DROP/CREATE, so you may need to write another script will re-enable specific permissions. This permission script would be the last to run. Our experience found more issues with ALTER verses DROP/CREATE semantics. YMMV

Answer 6

Couple different perspectives from my experience. In the Oracle world, everything was managed by "create" DDL scripts. As ahockley mentioned, one script for each object. If the object needs to change, its DDL script is modified. There's one wrapper script that invokes all the object scripts so that you can deploy the current DB build to whatever environment you want. This is for the main core create.

Obviously in a live application, whenever you push a new build that requires, say, a new column, you're not going to drop the table and create it new. You're going to do an ALTER script and add the column. So each time this kind of change needs to happen, there are always two things to do: 1) write the alter DDL and 2) update the core create DDL to reflect the change. Both go into source control, but the single alter script is more of a momentary point in time change since it will only be used to apply a delta.

You could also use a tool like ERWin to update the model and forward generate the DDL, but most DBAs I know don't trust a modeling tool to gen the script exactly the way they want. You could also use ERWin to reverse engineer your core DDL script into a model periodically, but that's a lot of fuss to get it to look right (every blasted time you do it).

In the Microsoft world, we employed a similar tactic, but we used the Red Gate product to help manage the scripts and deltas. Still put the scripts in source control. Still one script per object (table, sproc, whatever). In the beginning, some of the DBAs really preferred using the SQL Server GUI to manage the objects rather than use scripts. But that made it very difficult to manage the enterprise consistently as it grew.

If the DDL is in source control, it's trivial to use any build tool (usually ant) to write a deployment script.