Using Flyway to load data conditionally per environment

走远了吗. 提交于 2019-11-30 08:53:34

For future visitors, this is a solution for DB specific sql, but applies to data loading too. https://flywaydb.org/documentation/faq#db-specific-sql

You can set the flyway.locations=sql/common,sql/data property, and this can be set to different values with spring profiles(dev/test/prod), omitting the sql/data scripts on production.

If you are using maven, you can able to achieve it very easily through maven profiles concept. Please refer the following sample

pom.xml

<plugin>
    <groupId>com.googlecode.flyway</groupId>
    <artifactId>flyway-maven-plugin</artifactId>
    <version>2.3</version>
    <configuration>
        <url>jdbc:sqlserver://${db.hostname};databaseName=${db.name}</url>
        <user>${db.username}</user>
        <password>${db.password}</password>
        <initVersion>0</initVersion>
        <initDescription>Base Migration</initDescription>
        <table>Changelog_testproject</table>
        <locations>
           <location>filesystem:${sql.file.path}</location>
        </locations>
    </configuration>
</plugin>

<profiles>
    <profile>
        <id>dev</id>
        <properties>
            <profile.name>dev</profile.name>
            <sql.file.path>${basedir}/deploy/dev/sqldelta/sqlserver</sql.file.path> 
            <db.hostname>127.0.0.1:1433</db.hostname>
            <db.name>dev</db.name>
            <db.username>dev</db.username>
            <db.password>devadmin</db.password>
        </properties>
    </profile>
    <profile>
        <id>test</id>
        <properties>
            <profile.name>test</profile.name>
            <sql.file.path>${basedir}/deploy/test/sqldelta/sqlserver</sql.file.path>  
            <db.hostname>127.0.0.1:1433</db.hostname>
            <db.name>test</db.name>
            <db.username>test</db.username>
            <db.password>testadmin</db.password>
        </properties>
    </profile>
 </profiles>

Maven profiles did not give me the flexibility I wanted. I came up with a strategy that uses ant to merge files. This allows me to have common scripts and then included additional data depending on the deploy type eg. prod, dev.

This is my project structure:

├── build.xml
├── database.properties
├── database.properties.template
├── lib
│   └── ant-contrib-1.0b3.jar
├── pom.xml
└── sql
    ├── common
    │   ├── V1.0__.sql
    │   ├── V1.2.1__.sql
    │   └── V1.3__.sql
    ├── dev
    │   ├── V1.0__.sql
    │   └── V1.3__.sql
    └── prod
        └── V1.0__.sql

database.properties.template file in the root folder, before running this has to manually be copied to database.properties and the username and password can be entered. database.properties should be ignored by VCS so that passwords don't end up in the repo.

deployType scripts are merged into the src/main/resources/db/migrate directory, here is the ant script that does that, note that scripts that are to be merged have the same name:

<project name="data" default="prepareSql">

    <property file="database.properties" />
    <property name="destDir" value="src/main/resources/db/migration" />

    <echo message="Deploy type: ${deployType}"/>

    <taskdef resource="net/sf/antcontrib/antcontrib.properties">
      <classpath>
        <pathelement location="lib/ant-contrib-1.0b3.jar"/>
      </classpath>
    </taskdef>

    <target name="prepareSql">
        <!-- ensure the dest dir exists -->
        <mkdir dir="${destDir}"/>

        <!-- clear out the dest dir -->
        <delete>
            <fileset dir="${destDir}">
                <include name="*" />
            </fileset>
        </delete>

        <!-- append the deploy type files to the common files, delegate to the append target -->
        <foreach target="append" param="file">
            <fileset dir="sql/common" casesensitive="yes">
                <include name="*" />
            </fileset>
        </foreach>
    </target>

    <target name="append">
        <basename property="basename" file="${file}" />
        <property name="destFile" value="${destDir}/${basename}"/>
        <echo message="Appending ${file} to ${destFile}" />

        <concat destfile="${destFile}" >
            <filelist dir="sql/common" files="${basename}" />
            <filelist dir="sql/${deployType}" files="${basename}" />
        </concat>
    </target>
</project>

This ant file is executed by maven, here is the pom config:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>data</groupId>
    <artifactId>data</artifactId>
    <version>1.1-SNAPSHOT</version>
    <packaging>pom</packaging>
    <name>data</name>

    <properties>
        <sqlBaseDir>filesystem:${basedir}/src/main/resources/</sqlBaseDir>
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-antrun-plugin</artifactId>
                <version>1.7</version>
                <executions>
                    <execution>
                        <id>mergeScripts</id>
                        <phase>validate</phase>
                        <inherited>false</inherited>
                        <configuration>
                            <target>
                                <ant antfile="build.xml" target="prepareSql" />
                            </target>
                        </configuration>
                        <goals>
                            <goal>run</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.flywaydb</groupId>
                <artifactId>flyway-maven-plugin</artifactId>
                <version>3.2.1</version>
                <configuration>
                    <schemas>
                        <schema>common</schema>
                    </schemas>
                    <configFile>database.properties</configFile>
                    <table>flyway</table>
                    <locations>
                        <location>filesystem:${basedir}/src/main/resources/db/migration</location>
                    </locations>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.36</version>
        </dependency>
    </dependencies>
</project>

Should you require different data for different client distributions you could add dist directories into the sql directory and they could contain deployType sub directories.

Add the dist property to the database.properties.template file and then modify the append target in the build.xml to look like this:

<target name="append">
    <basename property="basename" file="${file}" />
    <property name="destFile" value="${destDir}/${basename}"/>
    <echo message="Appending ${file} to ${destFile}" />

    <concat destfile="${destFile}" >
        <filelist dir="sql/common" files="${basename}" />
        <filelist dir="sql/${deployType}" files="${basename}" />
            <filelist dir="sql/${dist}/common" files="${basename}" />
            <filelist dir="sql/${dist}/${deployType}" files="${basename}" />
    </concat>
</target>

You can use flyway placeholder:

Configure it on your enviroment config.properties:

flyway.placeholders.tableName=MY_TABLE

flyway.placeholders.name='Mr. Test'

Then, put it on your script: INSERT INTO ${tableName} (name) VALUES (${name});

I have used the flyway.locations too, but the placeholders is simpler than locations for simple changes.

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!