mirror of
https://github.com/django/django.git
synced 2024-11-29 14:46:18 +01:00
b7ea494d65
There was confusion about how to migrate data from third-party applications when you are going to uninstall the application later on. Thanks to Markus, Marten and Sergei for help and review.
335 lines
11 KiB
Plaintext
335 lines
11 KiB
Plaintext
===========================
|
||
Writing database migrations
|
||
===========================
|
||
|
||
This document explains how to structure and write database migrations for
|
||
different scenarios you might encounter. For introductory material on
|
||
migrations, see :doc:`the topic guide </topics/migrations>`.
|
||
|
||
.. _data-migrations-and-multiple-databases:
|
||
|
||
Data migrations and multiple databases
|
||
======================================
|
||
|
||
When using multiple databases, you may need to figure out whether or not to
|
||
run a migration against a particular database. For example, you may want to
|
||
**only** run a migration on a particular database.
|
||
|
||
In order to do that you can check the database connection's alias inside a
|
||
``RunPython`` operation by looking at the ``schema_editor.connection.alias``
|
||
attribute::
|
||
|
||
from django.db import migrations
|
||
|
||
def forwards(apps, schema_editor):
|
||
if not schema_editor.connection.alias == 'default':
|
||
return
|
||
# Your migration code goes here
|
||
|
||
class Migration(migrations.Migration):
|
||
|
||
dependencies = [
|
||
# Dependencies to other migrations
|
||
]
|
||
|
||
operations = [
|
||
migrations.RunPython(forwards),
|
||
]
|
||
|
||
You can also provide hints that will be passed to the :meth:`allow_migrate()`
|
||
method of database routers as ``**hints``:
|
||
|
||
.. snippet::
|
||
:filename: myapp/dbrouters.py
|
||
|
||
class MyRouter(object):
|
||
|
||
def allow_migrate(self, db, app_label, model_name=None, **hints):
|
||
if 'target_db' in hints:
|
||
return db == hints['target_db']
|
||
return True
|
||
|
||
Then, to leverage this in your migrations, do the following::
|
||
|
||
from django.db import migrations
|
||
|
||
def forwards(apps, schema_editor):
|
||
# Your migration code goes here
|
||
...
|
||
|
||
class Migration(migrations.Migration):
|
||
|
||
dependencies = [
|
||
# Dependencies to other migrations
|
||
]
|
||
|
||
operations = [
|
||
migrations.RunPython(forwards, hints={'target_db': 'default'}),
|
||
]
|
||
|
||
If your ``RunPython`` or ``RunSQL`` operation only affects one model, it's good
|
||
practice to pass ``model_name`` as a hint to make it as transparent as possible
|
||
to the router. This is especially important for reusable and third-party apps.
|
||
|
||
Migrations that add unique fields
|
||
=================================
|
||
|
||
Applying a "plain" migration that adds a unique non-nullable field to a table
|
||
with existing rows will raise an error because the value used to populate
|
||
existing rows is generated only once, thus breaking the unique constraint.
|
||
|
||
Therefore, the following steps should be taken. In this example, we'll add a
|
||
non-nullable :class:`~django.db.models.UUIDField` with a default value. Modify
|
||
the respective field according to your needs.
|
||
|
||
* Add the field on your model with ``default=uuid.uuid4`` and ``unique=True``
|
||
arguments (choose an appropriate default for the type of the field you're
|
||
adding).
|
||
|
||
* Run the :djadmin:`makemigrations` command. This should generate a migration
|
||
with an ``AddField`` operation.
|
||
|
||
* Generate two empty migration files for the same app by running
|
||
``makemigrations myapp --empty`` twice. We've renamed the migration files to
|
||
give them meaningful names in the examples below.
|
||
|
||
* Copy the ``AddField`` operation from the auto-generated migration (the first
|
||
of the three new files) to the last migration and change ``AddField`` to
|
||
``AlterField``. For example:
|
||
|
||
.. snippet::
|
||
:filename: 0006_remove_uuid_null.py
|
||
|
||
# -*- coding: utf-8 -*-
|
||
# Generated by Django A.B on YYYY-MM-DD HH:MM
|
||
from __future__ import unicode_literals
|
||
|
||
from django.db import migrations, models
|
||
import uuid
|
||
|
||
|
||
class Migration(migrations.Migration):
|
||
|
||
dependencies = [
|
||
('myapp', '0005_populate_uuid_values'),
|
||
]
|
||
|
||
operations = [
|
||
migrations.AlterField(
|
||
model_name='mymodel',
|
||
name='uuid',
|
||
field=models.UUIDField(default=uuid.uuid4, unique=True),
|
||
),
|
||
]
|
||
|
||
* Edit the first migration file. The generated migration class should look
|
||
similar to this:
|
||
|
||
.. snippet::
|
||
:filename: 0004_add_uuid_field.py
|
||
|
||
class Migration(migrations.Migration):
|
||
|
||
dependencies = [
|
||
('myapp', '0003_auto_20150129_1705'),
|
||
]
|
||
|
||
operations = [
|
||
migrations.AddField(
|
||
model_name='mymodel',
|
||
name='uuid',
|
||
field=models.UUIDField(default=uuid.uuid4, unique=True),
|
||
),
|
||
]
|
||
|
||
Change ``unique=True`` to ``null=True`` -- this will create the intermediary
|
||
null field and defer creating the unique constraint until we've populated
|
||
unique values on all the rows.
|
||
|
||
* In the first empty migration file, add a
|
||
:class:`~django.db.migrations.operations.RunPython` or
|
||
:class:`~django.db.migrations.operations.RunSQL` operation to generate a
|
||
unique value (UUID in the example) for each existing row. For example:
|
||
|
||
.. snippet::
|
||
:filename: 0005_populate_uuid_values.py
|
||
|
||
# -*- coding: utf-8 -*-
|
||
# Generated by Django A.B on YYYY-MM-DD HH:MM
|
||
from __future__ import unicode_literals
|
||
|
||
from django.db import migrations, models
|
||
import uuid
|
||
|
||
def gen_uuid(apps, schema_editor):
|
||
MyModel = apps.get_model('myapp', 'MyModel')
|
||
for row in MyModel.objects.all():
|
||
row.uuid = uuid.uuid4()
|
||
row.save()
|
||
|
||
class Migration(migrations.Migration):
|
||
|
||
dependencies = [
|
||
('myapp', '0004_add_uuid_field'),
|
||
]
|
||
|
||
operations = [
|
||
# omit reverse_code=... if you don't want the migration to be reversible.
|
||
migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop),
|
||
]
|
||
|
||
* Now you can apply the migrations as usual with the :djadmin:`migrate` command.
|
||
|
||
Note there is a race condition if you allow objects to be created while this
|
||
migration is running. Objects created after the ``AddField`` and before
|
||
``RunPython`` will have their original ``uuid``’s overwritten.
|
||
|
||
.. _non-atomic-migrations:
|
||
|
||
Non-atomic migrations
|
||
~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
.. versionadded:: 1.10
|
||
|
||
On databases that support DDL transactions (SQLite and PostgreSQL), migrations
|
||
will run inside a transaction by default. For use cases such as performing data
|
||
migrations on large tables, you may want to prevent a migration from running in
|
||
a transaction by setting the ``atomic`` attribute to ``False``::
|
||
|
||
from django.db import migrations
|
||
|
||
class Migration(migrations.Migration):
|
||
atomic = False
|
||
|
||
Within such a migration, all operations are run without a transaction. It's
|
||
possible to execute parts of the migration inside a transaction using
|
||
:func:`~django.db.transaction.atomic()` or by passing ``atomic=True`` to
|
||
``RunPython``.
|
||
|
||
Here's an example of a non-atomic data migration that updates a large table in
|
||
smaller batches::
|
||
|
||
import uuid
|
||
|
||
from django.db import migrations, transaction
|
||
|
||
def gen_uuid(apps, schema_editor):
|
||
MyModel = apps.get_model('myapp', 'MyModel')
|
||
while MyModel.objects.filter(uuid__isnull=True).exists():
|
||
with transaction.atomic():
|
||
for row in MyModel.objects.filter(uuid__isnull=True)[:1000]:
|
||
row.uuid = uuid.uuid4()
|
||
row.save()
|
||
|
||
class Migration(migrations.Migration):
|
||
atomic = False
|
||
|
||
operations = [
|
||
migrations.RunPython(gen_uuid),
|
||
]
|
||
|
||
The ``atomic`` attribute doesn't have an effect on databases that don't support
|
||
DDL transactions (e.g. MySQL, Oracle).
|
||
|
||
Controlling the order of migrations
|
||
===================================
|
||
|
||
Django determines the order in which migrations should be applied not by the
|
||
filename of each migration, but by building a graph using two properties on the
|
||
``Migration`` class: ``dependencies`` and ``run_before``.
|
||
|
||
If you've used the :djadmin:`makemigrations` command you've probably
|
||
already seen ``dependencies`` in action because auto-created
|
||
migrations have this defined as part of their creation process.
|
||
|
||
The ``dependencies`` property is declared like this::
|
||
|
||
from django.db import migrations
|
||
|
||
class Migration(migrations.Migration):
|
||
|
||
dependencies = [
|
||
('myapp', '0123_the_previous_migration'),
|
||
]
|
||
|
||
Usually this will be enough, but from time to time you may need to
|
||
ensure that your migration runs *before* other migrations. This is
|
||
useful, for example, to make third-party apps' migrations run *after*
|
||
your :setting:`AUTH_USER_MODEL` replacement.
|
||
|
||
To achieve this, place all migrations that should depend on yours in
|
||
the ``run_before`` attribute on your ``Migration`` class::
|
||
|
||
class Migration(migrations.Migration):
|
||
...
|
||
|
||
run_before = [
|
||
('third_party_app', '0001_do_awesome'),
|
||
]
|
||
|
||
Prefer using ``dependencies`` over ``run_before`` when possible. You should
|
||
only use ``run_before`` if it is undesirable or impractical to specify
|
||
``dependencies`` in the migration which you want to run after the one you are
|
||
writing.
|
||
|
||
Migrating data when replacing an external app
|
||
=============================================
|
||
|
||
If you plan to move from one external application to another one with a similar
|
||
data structure, you can use a data migration. If you plan to remove the old
|
||
application later, you will need to set the ``dependencies`` property
|
||
dynamically. Otherwise you will have missing dependencies once you uninstall
|
||
the old application.
|
||
|
||
.. snippet::
|
||
:filename: myapp/migrations/0124_ensure_dependencies.py
|
||
|
||
from django.apps import apps as global_apps
|
||
from django.db import migrations
|
||
|
||
def forward(apps, schema_editor):
|
||
"""
|
||
see below
|
||
"""
|
||
|
||
class Migration(migrations.Migration):
|
||
|
||
operations = [
|
||
migrations.RunPython(forward, migrations.RunPython.noop),
|
||
]
|
||
dependencies = [
|
||
('myapp', '0123_the_previous_migration'),
|
||
('new_external_app', '0001_initial'),
|
||
]
|
||
|
||
if global_apps.is_installed('old_external_app'):
|
||
dependencies.append(('old_external_app', '0001_initial'))
|
||
|
||
In your data migration method, you will need to test for the old application
|
||
model:
|
||
|
||
.. snippet::
|
||
:filename: myapp/migrations/0124_ensure_dependencies.py
|
||
|
||
def forward(apps, schema_editor):
|
||
try:
|
||
OldModel = apps.get_model('old_external', 'OldModel')
|
||
except LookupError:
|
||
return
|
||
|
||
NewModel = apps.get_model('new_external', 'NewModel')
|
||
NewModel.objects.bulk_create(
|
||
NewModel(new_attribute=old_object.old_attribute)
|
||
for old_object in OldModel.objects.all()
|
||
)
|
||
|
||
This way you can deploy your application anywhere without first installing
|
||
and then uninstalling your old external dependency. If the old external
|
||
dependency is not installed when the migration runs it will just do nothing
|
||
instead of migrating the data.
|
||
|
||
Please take also into consideration what you want to happen when the migration
|
||
is unapplied - you could either do nothing or remove some or all data from
|
||
the new application model; adjust the second argument of the
|
||
:mod:`~django.db.migrations.operations.RunPython` operation accordingly.
|