Best Practices for Django Migrations

From the official Django documentation:

Migrations are Django’s way of propagating changes you make to your models (adding a field, deleting a model, etc.) into your database schema. They’re designed to be mostly automatic, but you’ll need to know when to make migrations, when to run them, and the common problems you might run into.

This article aims to lay out some good practices related to database Django migrations i.e. what to do and what not to do and how to generally go about working with them.

What Are Django Migrations?

Django has been designed to best work with a relational database with SQL under the hood. Django provides its own fantastic ORM (Object-Relational Mapper) which abstracts away almost all (if not all) SQL from sight. The API the Django ORM provides for dealing with the database is easy to use, efficient, and covers a lot of ground as far as its purpose goes.

But to enable working with a relational database through an OO (Object-Oriented) API, the necessity of database migrations arises. Simply put, relational databases need their structure defined before you are able to utilize them. This pre-decided structure or organization of future data is also called Database Schema.

Schema creation, deletion, and updation are what migrations do. Note that we are specifically talking about schema migrations here, we will talk about data migrations later on in the article.

As the documentation says, this process is mostly automatic with little human intervention required. You simply create or edit your Django models like normal Python classes and Django’s makemigrations command generates the equivalent of the corresponding DDL (Data Definition Language; a subset of SQL commands used to create, modify, and delete database structures). Then the migrate command runs the generated DDL and the database structure is hence updated without ever needing to work with any SQL. Obviously, there are nuances to the process but this is the gist of it.

I hear you, enough with the introduction. Let’s get on to what the title says. Here are some good practices related to migrations in Django.

Check The Generated Migrations

While Django is pretty good at putting in migrations what you meant when editing a model, it can sometimes get confused as the process is not 100% perfect even though it tries to ask you for clarification in certain cases. For complex changes, it might not be detecting what you expect it to generate. You should also apply the generated migrations right away to confirm they work as expected.

Migrations Should be Version Controlled

Besides the fact that you can verify the generated migrations as mentioned above, another reason for having two separate commands to make and apply migrations is that you are supposed to commit migrations to your VCS and use them throughout your development team and in your staging or production server. For the easiest time around, only a single developer should make changes to a specific model at a time and then make a single commit for all the related migrations to avoid conflicts. This way, other developers will pull in migrations from other members of the team and won’t need to generate their own migrations, which can include fixing the generated migrations or answering the questions asked by the makemigrations command. Migrations should be a shared resource between all the local, staging, and production setups to have everybody on the same page regarding the state of the database schema. 

This plays a crucial role in avoiding any annoying discrepancies and in avoiding a common phrase used by developers in the field: “…but it worked on my machine!”.

source: donthitsave.com

Give Names to Your Migrations

Migrations can be thought of as a VCS for the database schema where makemigrations is the equivalent of committing your changes. Git makes giving a useful description to every commit compulsory. So we should just as well give names to our migrations with the –name flag to makemigrations command. While this is not absolutely necessary, it can certainly help when looking back at a big project for a certain file. 

After all, the auto-generated names given to each migration file are purely for the developer’s reference. Why not go one step further and give them appropriate meaningful names? Django just cares that each file has a different name and is in a valid migration format. The order of running the migrations is determined through each migration specifying its dependency on any previous migrations. Meaningfully naming things is just plain helpful in programming even though it takes a short extra minute. From Twitter:

There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors.

Your Database Backend Affects Migrations

Your migrations behave differently with different database backends e.g. PostgreSQL, MySQL, SQLite, etc. PostgreSQL is the most flexible in modifying its schemas. MySQL lacks support for transactions around schema alteration operations so to retry a failed migration operation, you will need to manually undo the partially applied changes. Even worse is SQLite which has very little built-in schema alteration support so it’s on Django to emulate it for you. 

Keep this in mind whenever working on a Django project and you will have fewer surprises.

Use Data Migrations

From the documentation: As well as changing the database schema, you can also use migrations to change the data in the database itself, in conjunction with the schema if you want.

Migrations that alter data are usually called “data migrations”; they’re best written as separate migrations, sitting alongside your schema migrations.

Django can’t automatically generate data migrations for you, as it does with schema migrations, but it’s not very hard to write them. Consult the documentation to learn how to do it.

An example scenario that the Django developers present where data migrations are useful is when you have originally created a model with first_name and last_name fields but you later realize that not everybody has a first and a last name. So you decide to combine the existing first and last names in the database into a single full_name field. A simple data migration can be written to accomplish this instead of writing raw SQL.

Consider Squashing

When the number of migration files gets too large, maybe in the hundreds, it can make sense to squash groups of related migrations into single combined migrations. Squashed migrations have the same combined effect as the un-squashed ones they consist of. You can choose which migrations to include in a squash and you can also give a descriptive name to the resultant squashed migration. All this is done with the squashmigrations command.

This is in keeping with our VCS analogy for migrations that in a VCS, multiple commits can be squashed together.

More reading

Thirsty for more? Here’re some more resources for detailed reading:

• https://docs.djangoproject.com/en/4.2/topics/migrations/#module-django.db.migrations
• https://nextlinklabs.com/resources/insights/naming-django-migrations-improving-projects
• https://realpython.com/django-migrations-a-primer/
• https://realpython.com/digging-deeper-into-migrations/#playing-chess-with-django
• https://martinfowler.com/bliki/TwoHardThings.html