Drupal 7 to Drupal 8 migration with configuration entities

Submitted by christophe on Wed, 04/04/2018 - 20:38

Since Drupal 8.5.0, the migrate system is stable, so this is a great time to review the migration tools provided by the community.

This post does not cover migration from the UI, it focuses on partially customized migration that are runned with Drush. In most cases, you will probably want to review the content model a bit (you know, that content type or field machine name that does not actually reflect the truth since its creation, ...).

The idea behind is to delegate the heavy lifting to Migrate Upgrade for migration template generation, then apply content model changes if needed.

Prepare your setup

Require dependencies
The needed tools are quite obvious, the only tricky part for the moment is getting the right versions.

# Require Drush 8 https://www.drupal.org/project/migrate_upgrade/issues/2915726
composer require drush/drush:~8.0
composer require drupal/migrate_upgrade:~3.0
composer require drupal/migrate_tools:~4.0
composer require drupal/migrate_plus:~4.0

Then enable them.

drush en migrate_tools migrate_upgrade

Add Drupal 7 database references
Edit your Drupal 8 settings file, so in this example, you will have to import your Drupal 7 SQL dump in a separate d7 database.

// Database entry for `drush migrate-upgrade --configure-only`
$databases['upgrade']['default'] = array (
  'database' => 'd7',
  'username' => 'root',
  'password' => 'root',
  'prefix' => '',
  'host' => 'localhost',
  'port' => '3306',
  'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
  'driver' => 'mysql',
// Database entry for `drush migrate-import`
$databases['migrate']['default'] = array (
  'database' => 'd7',
  'username' => 'root',
  'password' => 'root',
  'prefix' => '',
  'host' => 'localhost',
  'port' => '3306',
  'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
  'driver' => 'mysql',
// Database entry for `drush migrate-import`
$databases['drupal_7']['default'] = array (
  'database' => 'd7',
  'username' => 'root',
  'password' => 'root',
  'prefix' => '',
  'host' => 'localhost',
  'port' => '3306',
  'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
  'driver' => 'mysql',

Get the Drupal 7 files
A good idea would be to import files on your development server with rsync, so you can get the changed files only before rolling a migration update.


# Get files on your local dev environment from the production server
rsync -avzh my_user@drupal_7_server:/var/www/my_d7_site/docroot/sites/default/files/ /var/www/my_d8_site/web/sites/default/files/d7/public_files/docroot/sites/default/files/

Get the migration templates

This step will generate migration configuration entities in the active configuration store.

You can make a database backup first, as it could be needed before enabling your custom migration module (to avoid a PreExistingConfigException). Note that restarting from a backup is the "hacking around" method, you may want to rename each migration template instead, after having exported them in your custom module. See below.

drush sql-dump > my_drupal_8.sql
drush migrate-upgrade --configure-only

Create a custom module and the configuration directory.

# Follow Drupal Console steps, naming the module my_migrate in this example.
drupal generate:module
# Create then the config/install directory in your module.
mkdir -p my_migrate/config/install

Export site configuration generated by drush migrate-upgrade --configure-only

drush config-export --destination=/tmp/migrate

Then copy it in the custom migration directory. Do not include migrate_plus.migration_group.default.yml

cp /tmp/migrate/migrate_plus.migration.* /tmp/migrate/migrate_plus.migration_group.migrate_*.yml /path/to/my_migrate/config/install/

Remove then unnecessary templates (entities that will not be part of your migration). When removing templates, make sure that they are not required in the migration_dependencies section. If you remove dependencies, make sure that they have been migrated in another way.


The migration template migrate_plus.migration.upgrade_d7_node_article.yml has the following requirements:

    - upgrade_d7_user
    - upgrade_d7_node_type

For some reasons, you could have created content types manually and want to get rid of the upgrade_d7_node_type template.


Optionally customize your migration with machine name changes, source / process plugins or hook_migrate_prepare_row() / hook_migrate_MIGRATION_ID_prepare_row().

Restart then from your Drupal 8 database backup, or just change your migration templates file name, uuid, id and group for each yml file. Just keep in mind that while renaming your migration templates (configuration entities) id, you will have to rename the required and optional dependencies as well.


Run migration with Drush

Once done with your migration templates, enable your new module.

drush en my_migrate

Start to test your migration by id, with dependencies that are required by other entities first (role, user, file, taxonomy term, ...).

# Check the migrate status first, by id
drush ms upgrade_d7_user_role
# Migrate import by id
drush mim upgrade_d7_user_role


Include changes that were made to entities (new or modified ones).

drush mim upgrade_d7_user --update


drush mim upgrade_d7_file --limit="10 items"

Display migration messages

drush mmsg upgrade_d7_file

Errors will be logged in these messages, so it could be nice to redirect them into a log file.

drush mmsg upgrade_d7_file > sites/default/files/migration_logs/upgrade_d7_file.txt


drush mr upgrade_d7_file

Other Drush migrate commands


Users and roles

# Migrate status of roles and users
drush ms upgrade_d7_user_role
drush ms upgrade_d7_user
# Migrate import for roles then users
drush mim upgrade_d7_user_role
drush mim upgrade_d7_user

Content and translation

# Migrate content type article + i18n translation
drush mim upgrade_d7_node_article
drush mim upgrade_d7_node_translation_article


Public and private files

If you have imported your files in /var/www/my_d8_site/web/sites/default/files/d7/public_files/docroot/sites/default/files/
modify the /my_migrate/config/install/migrate_plus.migration.upgrade_d7_file.yml file by setting the source_base_path to /var/www/my_d8_site/web/sites/default/files/d7/public_files/docroot

# For a legacy site that has /docroot/sites/default/files
source_base_path: /var/www/my_d8_site/web/sites/default/files/d7/public_files/docroot

Do the same in the /my_migrate/config/install/migrate_plus.migration.upgrade_d7_file_private.yml file according to your private files path (ideally outside of the docroot).


drush mim upgrade_d7_file
# And optionally
drush mim upgrade_d7_file_private

Applying changes after having enabled the custom migration module

Use the Configuration Development module or drush config-import --partial


Before running migration import, and depending on the database size, you may consider increasing memory_limit in your php.ini


Migrate API


Users and Roles

Nodes, merge entities, translation

File Entities and Media Entities


I don't have drupal console installed is there a composer/drush equivalent to this part:

# Follow Drupal Console steps, naming the module my_migrate in this example.
drupal generate:module
# Create then the config/install directory in your module.
mkdir -p my_migrate/config/install


There is the drush mb command, but it requires the Module Builder module.

You can use Composer to require Drupal Console.

composer require drupal/console:~1.0

But chances are it is already there if you are using a solution like Composer template for Drupal projects.

Otherwise, for the case of a migration with content entities only, you just need the following structure, still not a lot of work manually:

  • my_migrate.info.yml
  • my_migrate.module
  • config/install directory to store your migrate_plus.* yml files

But I still recommend using Drupal Console if you need to finetune your migration with e.g. custom process plugins, there are also helpers for that.


Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.