Prepping Your App for a Smooth v2 Migration

Last Updated October 2014

Pagoda Box v2 is here and it’s time to get your app ready for the move. This doc runs through steps you can take to make sure your app’s migration to v2 runs as smoothly as possible. Also be sure to check out the Things to Know About Moving to v2 doc and the documentation on Butler, our v2 migration tool.

This doc is split into two main sections: Version Agnostic Changes & Changes Unique to v2. After reading it, you should be familiar with:

  • Changes that will make the move to v2 run smoothly
  • Changes you should make after your app is migrated to v2
  • Enjoying the new Pagoda Box and never looking back

Version Agnostic Changes

These changes will not affect how your app runs on either Pagoda Box v1 or v2. They simply make the move to v2 run more smoothly. Feel free to push these changes to both your v1 and v2 app.

Populate Connection Credentials with Environment Variables

Benefit: Instant service connections when deployed to v2

In order for a code service to interact with a data service, it must know how to connect to and, if necessary, authenticate with that service. Connection credentials are unique to each service and since new services will be created when migrating to v2, the credentials will be different. Any apps with hard-coded connection credentials will not be able to connect to their data services when initially deployed to v2.

To allow your app to immediately connect to data services when deployed to v2, you need to use the auto-generated environment variables to populate your connection credentials. These are automatically created when data services are created on both v1 and v2. The naming convention for environment variables is as follows:

Pattern:
<COMPONENT-ID>_<FOUR-LETTER-CRED>

Environment Variables for db1:
DB1_HOST
DB1_PORT
DB1_USER
DB1_PORT
DB1_NAME

Environment Variables for cache1:
CACHE1_HOST
CACHE1_PORT

Here's a simple example of using environment variables to populate a Laravel database config:

Using Environment Variables in Laravel DB Config PHP

app/config/database.php

If you opt to hard-code your credentials, you’ll have to wait until after your app has been migrated to v2, then update your code with the new connection credentials and deploy the updated code to your app. Until doing this, your app won’t be able to connect to its data services.

Note: This also applies to the php_session_save_path in your Boxfile, but this change should only be pushed to your v2 app. More information below.

Lock Your Composer Dependencies

If using Composer, you will need to commit your composer.lock file to your app's repo. The Lock File "locks in" what versions of each dependency Composer will use. Without a lock file, Composer will iterate through every declared dependency, search through all available versions, check if each version is compatible with other dependencies, then download the most recent compatible version. This process is incredibly memory intensive.

Composer runs during the build phase of the deploy process. RAM is limited in build instances and, without a lock file, Composer will likely run out of memory and your deploy to v2 will fail.

Avoid Using MyISAM Tables

Benefit: Reliable Data Migration

MyISAM was originally the default table engine used by MySQL, but was later replaced with InnoDB, primarily because of MyISAM’s inability to handle concurrent queries. MyISAM is non-transactional, meaning if two queries try to modify the table at the same time, MyISAM can’t queue them and ends up locking the table.

When migrating your app to v2, this can become a problem. In the app migration process, we migrate your data from your v1 instance to a new instance on v2. If a MyISAM table is modified while data is being exported, it will lock the table, the export will get stuck, and an engineer will have to manually intervene in order for the migration to proceed.

There are valid use cases for MyISAM tables, but in general, they should only be used to store rarely-modified data. Some use MyISAM for its full-text search capability. If you fall in this group, you may not be able to switch to InnoDB before migrating. The good news is that in MySQL 5.6+, InnoDB supports full-text search, so MyISAM is no longer necessary. MySQL 5.6 is available on v2. More information below.

Consolidate Worker Components

Benefit: Cost Reduction on v2

Something we’ve seen a lot of on Pagoda Box v1 is users creating multiple worker components, each with a single unique task. This approach was inadvertently encouraged by our providing of a free instance for all worker components. This is no longer the case on v2. Components (services) no longer get free instances. The more worker services you have, the more expensive your app will be.

Having multiple workers running unique tasks is unnecessary. A single worker can perform many tasks. You just need to tell a worker what to do. This is typically done by pushing tasks to some kind of job queue and having the worker pull tasks from that queue. Where the queue is stored and how it’s managed completely depends on the app. Laravel has job queuing functionality built in. Here’s some other projects that provide job worker/job-queue functionality:

PHP-Resque: PHP Resque Worker (and Enqueue) - uses Redis as a job queue.

Iron.io: A third party job queuing service.

Remove Any Unused Components

Benefit: Cost Reduction on v2

Some apps simply have unused components laying around. On v2, these unused instances will increase the price of your app. Our migration tool will launch any components that exist in your v1 app in your v2 app. If you have components you’re not using, you should remove them before migrating to v2. Otherwise, you will be charged for them.

Removing Web Components

Currently, web components can’t be removed through the v1 dashboard. The functionality exists, but was never fully released due to dependency issues. If you have web components you’d like removed, let us know in a ticket, and we’ll remove it for you.

Purge Unnecessary Writable Storage

Benefit: Cost Reduction on v2, Migration Time Reduction

Something most users don’t realize is that we have never charged for writable storage in v1 apps. Because writable storage has been free, many users simply don’t realize just how much data they’re storing (TBs of data in some cases).

In v2, we’ve completely re-architected writable storage (now called “Network Storage”). Instead of using a centralized storage cluster, network storage is now a component (service) in your app and scaled just like you would a web or database service. The more data you have stored the more disk you’ll need in your network storage service. As disk size increases, so does the price of network storage.

Purging unnecessary data from your writable storage will do a few things: It will reduce the need for disk space and the cost of network storage in your v2 app. It will also reduce the time it will take to migrate writable storage from v1 to v2. The more data you have, the longer it will take to move.


Changes Unique to v2

Important: The following changes, if made, should only be pushed to your v2 app. These modifications are isolated to the Boxfile. While v2 is built to be Boxfile-backwards-compatible, v1 is not forward-compatible and pushing these changes to your v1 app will cause problems. To isolate these changes, we recommend creating a new git branch and only pushing that branch to your v2 app. Here’s a basic Git workflow:

Git Workflow for Changes Unique to v2 Terminal

  # Create & Switch to "pagoda-v2" Branch
  $ git checkout -b pagoda-v2
  
  # Add a Remote for Your v2 App
  $ git remote add v2 git@git.pagodabox.io:apps/app-name.git
  
  # Add & Commit changes to your "v2" branch
  # Push Your "pagoda-v2" Branch to the "master" Branch of Your v2 App
  # (Only the "master" Branch Deploys on Pagoda v2)
  $ git add .
  $ git commit -m “updated boxfile with v2 changes”
  $ git push v2 pagoda-v2:master

Maintaing the two separate branches will allow you to continue pushing to both your v1 and v2 apps. If you make changes on your v1 deploy branch, you can just merge them into your pagoda-v2 branch and push them to your v2 app. Once your app is fully migrated and you no longer need your v1 app, merge your pagoda-v2 branch into your master branch and rename your v2 remote.

Git Merge Workflow Terminal

  # Checkout pagoda-v2 Branch & Merge Changes into "master"
  $ git checkout pagoda-v2
  $ git merge master
  
  # After app is migrated, Checkout "master" branch and merge "pagoda-v2" branch
  $ git checkout master
  $ git merge pagoda-v2
  
  # Delete the "pagoda-v2" branch
  $ git branch -D pagoda-v2
  
  # Rename Your "v2" Remote to "pagoda"
  $ git remote rename v2 pagoda
  
  # Push your "master" Branch to Pagoda v2
  $ git push pagoda master

Now on to the changes…

Update PHP Session Paths in the Boxfile

In this blog post, we recommended using Redis for native PHP sessions instead of writable storage. We still stand by this recommendation, but you will need to update your php_session_save_path in your Boxfile to use Environment Variables (which can be used in v2 Boxfiles).

PHP Session Save Path Using Environment Variables YAML

/Boxfile

Remove Composer after_build Hooks

To use Composer on Pagoda Box v1, you had to run it in an after_build hook. On v2, Composer runs by default on PHP apps, making running it in an after_build hook unnecessary. In fact, leaving your Composer after_build hooks in your Boxfile will increase the length of your deploys as composer will run 2+ times. We recommend removing all Composer after_build hooks from your Boxfile.

While we provide a default command to run composer, it’s possible to customize it. More information is available in the Composer doc.

Another change in v2 worth noting – Code is no longer built separately for each web & worker service. Instead, your code is only built once, then deployed to all webs & workers. This means that Composer only needs to run once to load dependencies for all code services.

As mentioned above, one of the reasons some use MyISAM tables is because of their full-text search capabilities. In MySQL 5.6+, InnoDB now offers full-text search. To use MySQL 5.6, all you need to do is specify 5.6 as your mysql version in your Boxfile:

Specifying MySQL 5.6 in the Boxfile YAML

/Boxfile

Note: This change to your Boxfile will only work on Pagoda Box v2. If you switch from MyISAM to InnoDB on your v1 database, you will lose your ability to run full-text searches.

New v2 Boxfile Configs

v2 brings many new configurable options to the Boxfile. But know, in order to use any of the new configs, you'll have to update your entire Boxfile to match the v2 format. The Legacy Boxfile Translations doc shows what v1 configs need to be update and how they should be updated.

If you have any questions, suggestions, or corrections, let us know.