Workers

Last Updated October 2014

Workers allow you to offload processes from your web service and run them in the background. They are generally used for heavy or long-running processes like mail, billing, image processing, etc. After reading this doc, you should understand:

  • The basic infrastructure of workers
  • How to deploy workers
  • How to configure workers

Basic Infrastructure

Whenever code is deployed, workers are built using the same codebase as your web service. All code necessary to run your workers should be included in your codebase.

While workers are built using the same codebase as your web service(s), they have no web server or publicly accessilble end-point. Workers can only be accessed from within your app or through an SSH Connection. They're usually setup up to pull jobs from some kind of job queue. Queues can be handled in many different ways – using Redis' pub/sub functionality, storing jobs in your database or in your network filesystem, etc. It all depends on how you want to handle it.

Workers are scaled in exactly the same way as web services. You can increase the number of instances and/or the amount of RAM per instance. Which method of scaling is best depends on what your worker does. For tasks requiring a lot of RAM, you'll want to increase the amount of RAM per instance. For "light" but highly concurrent tasks, you're better off to increase your number of worker instances.

Using Workers

Workers allow you to run tasks in the background without timeouts. This makes them ideal for running processes that may affecting the performance of your site if run on a web service – anything from sending email, building reports, or processing images. Workers are most often accompanied by a job queue, from which they pull jobs. As mentioned above, job queues can come in many different forms.

Below are links to example PHP workers and some great tools to help with implementing PHP workers:

Barebones PHP Worker Example
php-resque: PHP Resque Worker (php worker with a Redis queue)
Laravel's Out-of-the-Box Job Queue

Deploying Workers

Deploying workers is done by including a worker service in your Boxfile.

Workers in the Boxfile YAML

  worker1:
    type: php
    version: 5.5
    exec: "php workers/mail.php"
/Boxfile

Configuring Workers

All configuration for workers is handled in the Boxfile. Which config options you use depends on worker "type," or what language is powering your worker. All that's needed to get a worker up and running is a "type" and "exec," the command to start your worker process.

exec

The exec is the command that will start your worker process.

exec YAML

  worker1:
    type: php
    exec: "php workers/mail.php"
/Boxfile

You Must Declare Your Executable

In your worker exec command, the executable must be declared. This can be done in one of two ways:

  1. Include the executable at the beginning of your command.
    Example: "php path/to/script.php"

  2. Declare in it your script with a shebang.

Language & Service Specific Settings

Each language available in worker services has its own unique Boxfile configuration options. There are also 3rd party services with configuration options available in the Boxfile. The following docs outline each:

PHP Settings

View the available PHP settings in the PHP Settings in the Boxfile doc.

New Relic Settings

New Relic is a 3rd Party service that provides deep analytics related to your application's performance. Available New Relic configuration settings are coverd in the New Relic doc.

If you have any questions, suggestions, or corrections, let us know.