Network Storage

Last Updated October 2014

Most web applications need to store files in the filesystem at some point. In a traditional, single-machine hosting environment, your app consists of a single instance of your source code, a single filesystem, all on a single server. On Pagoda Box, your app consists of spread across a network of servers. Network Storage allow all the different services of your app to write to a single, writable, shared filesystem. After reading this doc, you should be familiar with:

  • How network storage works in a distributed environment
  • How to specify network directories in your Boxfile
  • How to manage content inside your network directories

Understanding Writable Permissions in a Distributed Environment

The first thing to understand about apps on Pagoda Box is they consists of multiple web and/or worker instances spread across a network of servers, all working together as one. They do not reside on a single server (unless you have only one web instance). Each instance is isolated and totally unaware of other instances, which poses some interesting challenges as each instance tries to write to the filesystem.

For more information on the Pagoda Box environment, check out the Understanding the Environment doc.

The Problem

If each individual web/worker instance were able to write to its own filesystem, they could potentially write themselves out of sync. Each web request is routed to a different instance of your source code. If you were to upload an image, it would be written to the filesystem of the instance that receives the request, but not to the filesystem of the other instances. That means your next request could potentially be routed to an instance that knows nothing about the image you just uploaded, because it was written to the filesystem of another instance.

The Solution

On Pagoda Box, web and worker instances are read-only, but can be connected to a single, writable filesystem. This keeps them from writing themselves out of sync, but still allows them to write when necessary.

How Network Storage Works

Network storage services are writable filesystems available to your app's services. The filesystem is accessed across the network through network mounts.

Network Directories

Any directory in your application needing write-access can be specified as a network directory in your Boxfile. These directories are stored on your storage service and accessible to application instances. On deploy, network mounts are created over each specified directory, allowing all instances to connect and write to a single writable filesystem.

Network Directories Use the Same Filepath

Even though network directories are accessed across the network, you're app will still use the same filepath it would use if the directories were on the local filesystem.

Network Mounts

Network mounts essentially replace the directories within your code repo. They're still accessible through the same filepaths, but the contents of those directories inside your repo will not be visible to your live app. The network mounts link to the parallel directories in the writable filesystem.

Network Directories Will Not Sync with Your Repo

When you deploy, any files within your repo that are inside a directory specified as a network directory will not be accessible from your app. If there are files in your repo that you need, you will need to upload them directly into your storage service through either an ssh or sftp connection.

Creating Network Storage

Network storage services can be created through your dashboard or your Boxfile.

Through the Dashboard

To create a Network Storage service through your dashboard, click on "New Service," then click the Network Storage icon.

You're given the option to name your Network Storage service. If no name is provided, we will create one for you. Click "Create Service" to begin provisioning your Network Storage.

Through the Boxfile

To create a network storage service from your Boxfile, include a storage service with the type, "nfs". You can also provide a custom name for your service, but it's not required. If no name is provided, we will generate one for you.

Network Storage in the Boxfile YAML

    type: nfs
    name: custom_name

On your next deploy, your storage service will be created. Configuration options are covered in the Network Storage Settings in the Boxfile doc.

Specifying Network Directories in the Boxfile

In order to create network directories, you must first create a storage service to store them on. Network directories can be specified for both web and worker services in your Boxfile. Under each service that needs access to the directory, list directories that need to be writable. If you have multiple storage services, you must specify which service the network directory should be created on. If no storage service is specified, we will assume storage1.

Filepaths should be relative to the root of your repo.

Network Directories in the Boxfile YAML

    name: public
    type: nfs
    name: private
    type: nfs
        - public/uploads
        - public/tmp
        - private-images
        - private-images

Network Directories Shared by Services

If a shared asset directory needs to be accessed by multiple services (read or write access) they must be declared for each service.

Nested Network Directories Will Not Work

Do not list nested network directories in your Boxfile. If app/tmp needs to be writable, do not include both app and app/tmp as network directories. They will not work properly.

Managing Network Directories

All of your network directories can be managed through an SSH/SFTP connection as long as the SSH Access is enabled for your storage service. This can be done by clicking on your storage service in your dashboard to expand and manage its options. Under to the "Connection" tab, click the "Enable SSH Access" button. Once access is enabled, use the provided SSH connection details to securely connect to your network storage.

Opening an SSH Connection to Network Storage Terminal

  #Example SSH Connection
  $ ssh -p 1245 gopagoda@

If you prefer to use an SFTP client to connect, use the following connection credentials:

Connection Type: SFTP
Host: The provided SSH host
Port: The provided SSH port
Username: The provided SSH user
Password: Not Necessary — SFTP uses SSH authentication

Once connected, you'll have root access to your network directories.

Never Remove Network Directories While SSH'd In

When SSH'd in, never remove the directory mounted as a network directory (the directory you specified in your Boxfile). Removing this directory will invalidate the network mount that connects the directory to your code instances. Your app will lose it's ability to write it and any of its subdirectories. The only way to fix this is to redeploy your app or rebuild the affected code service(s).

Discouraged Uses

Network directories are a great tool, but there are ways we recommend they not be used.

Storing Sessions in Network Directories

Storing sessions in the filesystem generally isn't a very good idea, especially for large scale applications. Files take time to write and if a file is being updated multiple times a second, requests will begin to queue, potentially causing your network storage to become unresponsive. Storing sessions in network storage simply isn't a scalable solution.

We recommend storing sessions in a Redis service. Redis is great for handling sessions in large-scale, high-concurrency apps.

Storing Source Code in Network Directories

Storing source code or any critical app files (sessions, config files, dependencies, etc.) in network storage is highly discouraged for multiple reasons:

  1. No Version Control or Emergency Rollbacks - Network storage is not tracked by Git, so if any bugs are introduced through network storage, there is no easy way to roll back to a previous, bug free version. You'll have to manually edit the code.

  2. Decreased Performance - Connections to Network storage must travel accross the network. While minimal, network latency does increase response times to files stored in network directories. Running source code in your network storage negates the performance optimizations built into the Pagoda Box infrastructure.

  3. Code Caching + Writable Source Code = No Bueno - Opcode cachers like APC and eAccelerator are great for improving the performance of your app. But they do cause problems when multiple web or worker instances are running source code from network directories. When code is cached on Pagoda Box, we assume the code is read-only (which it should be). Once a code cacher caches the file, it never reads it again. So, when an instance caches code, it's cached until you redeploy. This can lead to odd behavior like code updates not being reflected in the live app or instances in the same app serving up different versions of the code.

  4. Network File System Reliability - Network directories are, as the name suggests, network file stores. If the network connection is interrupted in any way, source code stored in a network directory will be temporarily inaccessible, potentially causing an error in your application. In some cases, these errors can only be fixed by restarting or rebooting your web/worker service. Anything to which your app must have a constant connection should not be stored in network storage. Pagoda Box takes every precaution to make sure network connections are always stable, but interruptions can happen.

If you have any questions, suggestions, or corrections, let us know.