Why You Should Migrate Docker Container Volumes to Persistent Storage.

Docker containers are lightweight, efficient, and portable, but managing their data can become challenging as your applications scale. By default, Docker volumes provide a simple and effective way to persist container data. However, many developers and businesses eventually face the need to migrate their Docker volumes to persistent storage like bind mounts or external storage solutions.

When I first set up some of my testing Docker containers, I followed the default volume configuration from the documentation, which worked perfectly for getting started. However, when the time came to move these containers to a new server, I realised that managing scattered Docker-managed volumes could be challenging. I also started thinking about taking regular backups, and it became clear that having all container volumes organised in a single parent directory, with separate subdirectories for each container, would make things much easier. This blog post is a result of that experience—sharing the steps I took to migrate my Docker volumes to a structured directory setup, making backups and future migrations more straightforward.

In this blog post, we'll discuss who should migrate Docker container volumes to persistent storage and why it's essential, as well as address frequently asked questions about the process. Whether you’re a developer or a business owner looking for more robust data management, this guide is for you.

Who Should Consider Migrating Docker Volumes to Persistent Storage?

1. Developers Scaling Applications As your Dockerized applications grow, you may find that volumes are no longer sufficient for handling data. Migrating to persistent storage allows you to manage data outside the container lifecycle, ensuring scalability and better control.
2. Businesses Relying on Critical Data If your business depends on containerized applications like databases, CMS platforms (e.g., Ghost), or other data-driven services, migrating to persistent storage ensures that your data remains secure and accessible—even if containers are recreated.
3. Teams Needing Backup and Restore Options Persistent storage solutions, such as bind mounts, allow for easier backups, restores and migrations. This is especially critical for businesses running production applications.
4. Organisations Using Multi-Node Deployments For businesses using orchestration tools like Kubernetes or Docker Swarm, external persistent storage solutions ensure that data is accessible across nodes, providing high availability and fault tolerance.

Benefits of Persistent Storage

• Data Durability: Ensures data survives container restarts or deletions.
• Flexibility: Enables storage on external drives, cloud platforms, or shared networks.
• Ease of Backup: Facilitates routine backups without accessing container internals.
• Improved Security: Offers better control over data access and permissions.

How to Migrate Docker Volumes to Persistent Storage

Migrating volumes involves copying data from the existing Docker-managed volume to a persistent storage solution, like a bind mount or NFS share.

How to Find the Current Volumes of a Docker Setup

Before migrating Docker volumes to persistent storage, it’s important to identify the volumes currently in use by your containers. You can do this by inspecting your Docker setup. For example, if your container is named ghost you can run the following command:
If you do not knwo the name of your container you can also use the "ps" comand to view all runnig containers and use the Cotainer ID.

sudo ducker ps -a

Need to insert an example texhere

sudo docker inspect ghost

This will output a very detailed file about the container, including the Mounts section, which lists all volumes and bind mounts attached to the container. Here’s an example output snippet:

"Mounts": [
    {
        "Type": "volume",
        "Source": "docker_ghost",
        "Target": "/var/lib/ghost/content",
        "VolumeOptions": {}
    }
]

In this example:

• Source: docker_ghost is the name of the Docker-managed volume.
• Target: /var/lib/ghost/content is the path inside the container where the volume is mounted.

You can also list all volumes managed by Docker using:

sudo docker volume ls

This will display a list of all existing volumes. Identifying these volumes ensures you know exactly which data needs to be migrated and updated in your docker-compose.yml file.

How to Copy a Docker Volume to a New Location

If you’re new to Docker containers, moving data from a Docker-managed volume to a new folder might seem challenging, but it’s quite simple with the right steps. Let’s walk through how to copy data from a Docker volume (e.g., docker_ghost) to a new location on your computer, such as a folder called DockerContainers/ghost.

Step 1: Create the New Target Directory

Before copying, you need to create the folder where the data will be stored. For example, to store the data in a folder named ghost inside a parent folder called DockerContainers, run:

mkdir -p DockerContainers/ghost

The -p option ensures that both the parent folder (DockerContainers) and the target folder (ghost) are created if they don’t already exist.

Step 2: Copy Data from the Docker Volume

Docker volumes are not stored in easily accessible paths on your host machine, but you can use a temporary Docker container to copy their contents to the new folder.

Run the following command to copy the contents of the docker_ghost volume to the DockerContainers/ghost folder:

sudo docker run --rm \
    -v docker_ghost:/source \
    -v $(pwd)/DockerContainers/ghost:/destination \
    alpine sh -c "cp -r /source/. /destination/"

Here’s what this command does:

1. Creates a Temporary Container: The alpine image is a lightweight Linux container that will be used for this operation.
2. Mounts the Docker Volume: The -v docker_ghost:/source part attaches the docker_ghost volume to the container’s /source directory.
3. Mounts Your Target Directory: The -v $(pwd)/DockerContainers/ghost:/destination part attaches the folder DockerContainers/ghost on your computer to the container’s /destination directory.
4. Copies the Data: The cp -r /source/. /destination/ command inside the container copies all the files from the volume (/source) to your target folder (/destination).

Step 3: Verify the Data

Once the copying process is complete, check the contents of the new folder to ensure the data was successfully transferred:

ls -l DockerContainers/ghost

This command lists all the files in the DockerContainers/ghost folder, which should now contain the same data as the original docker_ghost volume.

Now that your Docker volumes have been successfully moved to the new mount (e.g., DockerContainers/ghost), you need to update your Docker configuration to use the new mount instead of the original Docker-managed volume. Here’s a step-by-step guide:

1. Update the docker-compose.yml File

Copy your existig docker compose file before making changes to the working Compose file

sudo cp docker-compose.yml docker-compose.yml.backup

Now you can modify your docker-compose.yml file to replace the Docker-managed volume with the new bind mount.

Example: Original Volume Configuration

volumes:
  - ghost:/var/lib/ghost/content

Updated Configuration with Bind Mount

Replace the volume with the path to your new directory (DockerContainers/ghost):

volumes:
  - ./DockerContainers/ghost:/var/lib/ghost/content

Explanation:

• ./DockerContainers/ghost: This is the path to the new folder where your data resides.
• /var/lib/ghost/content: This is the directory inside the container where Ghost stores its data. You do not have to changed this part of the file.

Save the changes to your docker-compose.yml file.

2. Stop and Recreate the Ghost Service

To apply the changes, you need to stop the existing container and restart it with the new configuration.

Stop the Service:

sudo docker compose down

Restart the Service:

sudo docker compose up -d

Docker will now use the data from the new bind mount (DockerContainers/ghost) instead of the old Docker-managed volume.

3. Verify That Ghost is Working

After restarting, check that your service is running correctly and using the data from the new bind mount:

1. Access Ghost: In this example I would visit your Ghost website (e.g., https://www.jimgogarty.com) and ensure your content is intact and all links and images are working.

Inspect the Running Container: Check that the new bind mount is attached to the container:

sudo docker inspect ghost

Look for the Mounts section in the output. It should show your new directory as the Source and /var/lib/ghost/content as the Target.

4. Clean Up the Old Volume (Optional)

If everything is working correctly and you no longer need the old Docker-managed volume, you can safely remove it,

Remove the old volume:

sudo docker volume rm docker_ghost

List all volumes to confirm the old volume name:

sudo docker volume ls

Note: Only do this after verifying that the new bind mount works and all data is intact.

5. Set Up Backups for the New Mount

Now that your data is stored in a bind mount, it’s easy to back up. Use a tool like tar to compress and save the data:

tar -czvf ghost_backup.tar.gz DockerContainers/

Store the backup in a safe location or automate backups using a cron job or a backup tool.

Migrating Docker container volumes to persistent storage is a crucial step for businesses and developers aiming for scalability, durability, and better data management. By understanding the benefits and following the right steps, you can ensure your applications and data remain resilient and secure.

If you’re looking for expert assistance with Docker migrations, reach out to JimGogarty.com. Our team is ready to help you achieve your IT goals with tailored solutions.

Contact Us

Have questions about migrating Docker volumes or need assistance with your IT infrastructure? Contact us today!