Manually Migrating Apps from Pagoda Box v1 to v2

Last Updated October 2014

Pagoda Box v2 is here and ready for your apps. This doc walks through manually migrating apps from v1 to v2, but to help more easily migrate your apps, we’ve also built a migration tool that will do most of the heavy lifting for you. The Migration Tool documentation walks through using it. If you’d rather migrate your app(s) manually, this doc will help you become familiar with:

  • Steps necessary for fully migrating apps from v1 to v2
  • Recommended data transfer methods
  • Moving to Pagoda Box v2 and never looking back

Prepare Your App for the Migration

This doc primarily focuses on the steps required to manually move your app to v2, but there are some changes you may need to make to your code in order for your app to function properly on v2. These changes are covered in the following docs:

Things to Know When Moving to Pagoda Box v2
Prepping Your App for a Smooth v2 Migration

v1 Boxfiles Will Work on v2

The v2 Boxfile parser has been built to be backwards compatible. It recognizes when a v1 (Legacy) Boxfile is being used and will automatically translate legacy configs into a format the v2 deployment system can understand. No Boxfile changes are necessary to get your app running on v2.

However, there are many more Boxfile config options available in v2 (as illustrated in the Boxfile Quick Reference). To use v2 configs, it’s all-or-nothing – either you need to update everything in your Boxfile to use v2 configs or stick with only Legacy configs. Boxfiles with both legacy and v2 configs will not work properly.

The Legacy Boxfile Translations doc shows exactly what legacy configs need to be updated and how they should be updated.

Add Your SSH Keys to Your V2 Account

In order to push code to your v2 app, you must first add your SSH key to your v2 user account. The Setting Up SSH doc walks through how.

Create Your App on V2

This is probably a given, but before you’ll be able to migrate your v1 app, you need to create a v2 app to migrate it to. This can be done from your dashboard homepage by clicking “Launch New App” or through the Pagoda CLI.

Launch an App

Once the app is created, you’ll be able to access the app’s dashboard, even before code has been pushed. This allows you, if necessary, to add team members, environment variables, your New Relic key, etc., before you ever actually deploy code.

Using Private Git Submodules

If your app uses private git submodules, you’ll need to add your app’s Public SSH key to your user account on the submodule’s hosting provider before you push your code. Otherwise the submodules will not get loaded correctly in the deploy process. App SSH keys are used to securely authenticate and connect with third party services and allow you to securely download private submodules. Your app’s public SSH key is found in your app dashboard under Dev Config > Public SSH Key.

Push Your Code to Your V2 App

With your app created, you’re ready to push your code. Instructions are provided in your dashboard.

Git Push Instructions

Chances are that if you’re migrating the app from v1 to v2, you already have a “pagoda” remote. The name of a remote is arbitrary and can be whatever you want. If you still want to maintain your v1 and v2 remotes, you’ll need to add a new git remote for your v2 repo using a different name.

Adding a Git Remote Terminal

  # Pattern
  $ git remote add  
  
  # Example
  $ git remote add v2 git@git.pagodabox.io:apps/appname.git
    

If you don’t need to continue maintaining your v1 remote, you can just change the git clone url of your existing pagoda remote:

Changing the Clone URL of a Git Remote Terminal

  # Pattern
  $ git remote set-url  
  
  # Example
  $ git remote set-url pagoda git@git.pagodabox.io:apps/appname.git

When you’re ready, go ahead and push to the master branch of your v2 remote.

Pushing Your Master Branch to Pagoda Box Terminal

  $ git push remote-name master

It should be noted that only the master branch will deploy on Pagoda Box v2. If the branch you’d like deployed is anything other than master, you will need to push that branch to the master branch of your pagoda remote. This is documented here.

Add Any Necessary Environment Variables

If you haven’t already, go ahead and add any environment variables required by your app. These can be included your Boxfile or added through the dashboard. The Environment Variables doc walks through both methods.

Migrate Your Data

This is one of the more critical steps of migrating your app to v2 and can be done in a number of different ways. The method outlined below is recommended simply because it allows you to complete the data migration without ever having to download the data locally or use the Pagoda Tunnel.

1. Scale Your v2 Database to a Similar Scale as Your v1 Database

Before you start the data migration process, you’ll want to scale your v2 database to be similar in size and resource allocation as your v1 database. This will help to ensure the data import will run successfully. While there isn’t a direct instance size correlation between v1 and v2, Instance sizes are similar. Go with the scale that’s the closest.

2. Make a Database Backup Through a v1 Cron Job

To quickly export and store a backup of your v1 database, you can use a cron job to run a mysqldump. v1 doesn't allow you to SSH directly into running instances and run native shell commands, but cron jobs will. It's admittedly a little "hacky", but it will get you what you need.

A pre-requisite to this approach is having at least one shared writable directory specified in your Boxfile. The cron job will store the backup file here.

Under the Cron Jobs tab in your v1 dashboard, click "Add a Job". Add the following command in the "Command" field, replacing the "path/to/writable_dir" with the correct path to your shared writable directory. This command also assumes you're migrating db1. If you're migrating another database, just update the environment variables used in the command with the correct component ID.

mysqldump Cron Command Terminal

  mysqldump -h $DB1_HOST -P $DB1_PORT $DB1_NAME -u $DB1_USER -p$DB1_PASS > path/to/writable_dir/data_migration.sql

Note: If you don't have any MyISAM tables in your database, we recommend including the "--single-transaction" flag in your mysqldump command. This will prevent tables from locking during the dump.

The schedule is required, but it's recommended you set it to something that won't run right away such as "0 0 1 1 *" (Once a year on January 1st). We also recommend adding yourself as an output email recipient, just in case there's an error. Save the cron job, then, once you're ready, click the "Run Now" button to run the backup.

v1 Cron Job Run Now

3. Enable SSH Access to Your v1 Shared Writable Storage

If SSH access to your writable storage is not already enabled, you will need to enable it. This is done under the Admin tab of your v1 app dashboard.

v1 Enable SSH Access

Check the box, provide a password and click “Save” in the lower right corner of your dashboard. With SSH access enabled, you’ll be able to access the backup file stored in your “backup” directory.

Ignore the “App Cannot Be Updated” Error

If, when enabling SSH access to writable storage, you get an error saying “The app could not be updated…”, ignore it. This is a known UI bug caused by a form validation error. Your app will update successfully.

4. Enable the SSH Portal on Your v2 Web Service

Pagoda Box v2 allows you to SSH directly into web services, but you must first enable the SSH portal. To do this, click on the service in your dashboard to expand its options. Under the "SSH" tab, click the "Enable SSH Portal" button.

Enable SSH Portal

5. SSH Into Your v2 Web Service

With the SSH portal enabled, go ahead and SSH into your v2 web service. The necessary SSH command is provided in your dashboard after the SSH portal is enabled.

6. Copy Your v1 Database Backup Into Your v2 Web Service

While SSH’d into your v2 web service, use “scp” to securely copy your v1 database backup file into the /tmp directory of your web instance. Be sure to replace "path/to/writable_dir" with the correct path to the writable directory where your backup file is stored.

scp Your v1 Data Backup Into Your v2 Web’s /tmp Directory Terminal

  $ scp v1-appname@pagodabox.com:shared/path/to/writable_dir/data_migration.sql /tmp
  
  # Enter your v1 writable storage SSH password when prompted

7. Import The v1 Backup File Into Your v2 Database

Now that the v1 data backup is on your v2 web instance’s local file system, you can import it into your v2 database. While SSH’d into your v2 web service, run the following command, replacing the backup file name with your backup file name:

Import the v1 Backup Into Your v2 Database Terminal

  $ mysql -h $DATABASE1_HOST -P DATABASE1_PORT -u $DATABASE1_USER -p$DATABASE1_PASS $DATABASE1_NAME < /tmp/data_migration.sql

This command uses environment variables to populate your database’s connection credentials and should work out of the box without changing anything other than the name of the sql file.

8. Remove the Backup File from the /tmp Directory

Once the import is complete, we highly recommended removing the backup file from the /tmp directory. While still SSH’d into your v2 web service, run the following command:

Remove the Backup File Terminal

  $ rm /tmp/data_migration.sql

You will probably also want to remove the backup file from your v1 writable storage. This can be done by simply SSH/SFTP'ing in and removing it.

Congratulations! Your data is migrated.

Migrate Your Writable Storage

If your v1 app has any shared_writable_dirs, those directories will need to be copied into a network storage service in v2 app. For more information about how writable storage (now “network storage”) works on v2, read through the Network Storage doc.

1. Create a Network Storage Service in Your v2 App

This can be done through your dashboard or your Boxfile. More details are provided in the Network Storage doc.

2. Enable SSH Access to Your v1 Writable Storage

If you haven’t done this already, go ahead and enable SSH access to your v1 writable storage. This is covered above.

3. Scale Your v2 Network Storage

It’s important that your network storage service has enough disk space to hold all of your transferred files. The base network storage instance has 4GB of disk. If you have more writable storage than this, you will need to scale to an appropriate size.

If you don’t know how much is currently stored in writable storage, submit a ticket with your app name and we’ll provide that information ( the du executable isn't available in writable storage ). Before you submit the ticket, make sure SSH access to your writable directories is enabled.

4. Enable the SSH Portal on Your v2 Network Storage

Pagoda Box v2 allows you to SSH directly into network services and run commands from within that context. You first need to enable the SSH portal. To do this, click on the network storage service in your dashboard to expand its options. Under the "SSH" tab, click the "Enable SSH Portal" button.

5. SSH Into v2 Network Storage Service

Once the SSH portal is enabled on your network storage service, SSH into the service using the command provided in your dashboard.

6. Copy Your v1 Writable Storage Into Your v2 Network Storage

While SSH’d into your v2 Network storage run the following command to copy all of your v1 writable storage into your v2 network storage. Simply replace “appname” with your v1 app name.

scp Your v1 Writable Storage Into Your v2 Network Storage Terminal

  $ scp -r appname@pagodabox.com:shared/* /data/
  
  # Enter your v1 writable storage SSH password when prompted

This will recursively copy all of your v1 writable directories into your v2 network storage. Depending on how much writable storage you actually have, this process may take a while. You will not need to update any system filepaths. This command will put your network directories right where your app is expecting them to be.

Add Cron Jobs

If you have any cron jobs on your v1 app, you’ll need to add them to your v2 app as well. This can be done either through the dashboard or the Boxfile. The Cron Jobs doc walks through both methods.

Add Mail Credentials on v2

If your v1 app uses the Pagoda Box mailer to send emails via the native php mail() function, you’ll need to provide your SMTP credentials in your v2 dashboard. This can be done under Dev Config > Mail in your app dashboard. The Sending Mail from Your App doc has more info.

Recreate SSL Certificates on v2

If your v1 app uses SSL certificates, those certificates will need to be recreated in your v2 app. Adding SSL certificates is covered in detail in the Adding SSL doc, so we won’t cover those steps here, but there are some things you should know.

Manually Migrating SSL Certificates Requires Downtime

For security reasons, Pagoda Box v2 will never accept SSL Private Keys from other services. We generate new private keys for every SSL certificate created on v2. Any existing certificate will need to be rekeyed to match the new private key. Once rekeyed, the corresponding v1 certificate it will no longer function and will not function until your DNS is pointed at the appropriate v2 IP.

For apps using SSL certificates, we highly recommend using the v2 Migration Tool. This copies the private key tied to your v1 SSL cert and transfers it with your certificate to your v2 app. When using the Migration Tool, certificates do not have to be rekeyed.

Point Your Domain to Your V2 App

Once you’re app is up and running on v2, you’re ready to point your domain to it. The Using Custom Domains & Routes doc walks through this process in detail.

Congratulations! Your App is Now Migrated

Once you’ve updated your DNS A-record to point to your v2 app and the change has fully propagated, the migration is complete. You can now decommission your v1 app and enjoy all the awesomeness that comes with the new Pagoda Box.

Table of Contents

Prepare Your App for the Migration

Add Your SSH Keys to Your V2 Account

Create Your App on V2

Push Your Code to Your V2 App

Add Any Necessary Environment Variables

Migrate Your Data

Migrate Your Writable Storage

Add Cron Jobs

Add Mail Credentials on v2

Recreate SSL Certificates on v2

Point Your Domain to Your V2 App

Congratulations! Your App is Now Migrated

If you have any questions, suggestions, or corrections, let us know.