Making migrations a breeze with Exodus

Behold! A disc-shaped device skewered on an axle that facilitates motion by rotation. I call it... the wheel! Allow this pinch of sarcasm to serve as acknowledgement that Exodus, the open-source tool introduced in this article, does not solve a new problem. While mature solutions to manage database schema changes in Java exist, I wanted to scratch my own itch and see how the sausage is made. Thus, I created my own partly to ensure the tool would be as lean as possible, and also as a learning exercise to enhance my familiarity with Spring under the hood — namely its application lifecycle, extensibility via plugins, and how it interacts with datasources.

It was in starting a new web backend with Spring recently that I was put off by the bloat, proprietary schemas, and premium plans being pushed in my face when considering tools to handle migrations. As I turned my nose up at different options I compiled a feature wishlist which boiled down to a handful of commandments:

• Thou shalt have a minimal footprint.
• Thou shalt store migrations as plain SQL files.
• Thou shalt avoid vendor lock-in that could hold your data hostage.

While the existing tools do the job just fine, once the possibility to build a migration runner of my own took hold I knew I had to give it a go. Surely it couldn't be that difficult, right?

Well, no, thankfully it was not. While there was the occasional moment that made me want to see if I could instead run migrations by bashing my head against the keyboard, the overall experience has been one of achievement in drawing the curtain back on some of Spring's internals & realising the desired functionality in an exceptionally lightweight package.

Project structure

The source code consists of database utilities and the migration runner's main loop. The former is made up of a number of raw SQL statements invoked by the latter. It also deals with bridging the gap between Spring and your project's database engine of choice. The main loop consists of a listener that hooks into Spring's event lifecycle in order to trigger migrations at the appropriate moment.

Exodus relies on dependency injection to know which database to run migrations on: the DatabaseUtils module will run code on the datasource provided by the Spring application it is attached to. This is provided by injection via the MigrationRunner's constructor, which is responsible for setting up a connection & statement for use throughout the Exodus instance's lifetime. Exodus concerns itself solely with the management of migrations, and as such the datasource to use is an implementation detail. DatabaseUtils provides ways to inspect the tables in the database, list applied migrations, and apply a pending migration.

The MigrationRunner implements Spring's ApplicationListener. As such, Exodus' main loop starts by overriding the ApplicationListener's onApplicationEvent(ContextStartedEvent event) method. In your web application's main() method it is necessary to emit a ContextStartedEvent that will trigger the routine to look for & apply migrations on start-up. Migrations should be written as plain .sql files stored in your Spring project's resources directory (see section on writing migrations below).

Version 1.0.0 is an MVP and fulfils the basic functionality I wanted to achieve while keeping true to the requirements listed above, coming in at just over 7 kilobytes once compiled into a JAR. Future versions will add code generation features, such as automatically creating rollback scripts for any processed migrations.

Add Exodus to a Spring project

1. Download the latest exodus.jar from GitHub and place this JAR in your Spring project's src/main/resources/lib/ directory.
2. Add Exodus as a dependency in your POM, taking care to set the version property as appropriate:

<dependency>
    <groupId>com.albertomh</groupId>
    <artifactId>exodus</artifactId>
    <version>1.0.0</version>
    <scope>system</scope>
    <systemPath>${basedir}/src/main/resources/lib/exodus-1.0.0.jar</systemPath>
</dependency>

3. Using the sample application entrypoint as a guide, do the following:
   • Pass scanBasePackages = {"com.albertomh.exodus"} as a parameter to the @SpringBootApplication decorator.
   • Instantiate an application context with SpringApplication.run(YourApplication.class, args).
   • Call applicationContext.start() inside main() to emit a ContextStartedEvent. This is the cue for Exodus to run any pending migrations.

These three steps are all that is needed to add Exodus to a project — you can now start writing migrations.

Write migrations

Exodus will pick up any .sql files you place under src/main/resources/db/migration/ in your Spring application. The following two best practices are encouraged (but not enforced at runtime):

• Subdivide db/migration/ into directories named after the year the migrations they hold were written in.
• Have migrations follow the naming convention YYYY-MM-DD_HH.MM__<MODULE>__<CHANGE>.sql where <MODULE> is a subdivision of your app's functionality and <CHANGE> is a concise summary of the change enacted by the migration. For instance: 1970-01-01_09.00__auth__create-user.sql.

On its first run Exodus will create a _schema_migration table to keep track of the migrations that have been applied. On subsequent runs, if a valid migration is found at any depth within db/migration/, the table will be queried and the migration applied if it is not listed therein.

Listen for migrations to finish

After every run Exodus will publish a custom Spring event, the MigrationCompleteEvent. You can listen for this event in order to execute code after all migrations have been applied. Have your class implement Spring's ApplicationListener, overriding the default onApplicationEvent method:

import org.springframework.context.ApplicationListener;
import com.albertomh.exodus.event.MigrationCompleteEvent;

public class MyComponent implements ApplicationListener<MigrationCompleteEvent> {
    @Override
    public void onApplicationEvent(MigrationCompleteEvent event) {
        // Your logic here.
    }
}

This is the best way to run logic that depends on all migrations being applied first.

Hopefully you have found this a worthwhile read about Spring internals and creating plugins for the framework. If you wish to contribute to or fork Exodus check out the project on GitHub and consult the documentation on building, testing, and cutting a release.

Main image made by remixing Bootstrap Icons (MIT license).
Dropcap background: Watercolour for Printed Fabric: Wey (1882-1883) by William Morris, licensed under CC0.