Failed to import volume with error container path is already in use

Overview

There are volumes that are unable to be imported by ComputeStacks, and the error message is “Container path is already in use.”

Cause

Volumes should always be defined in ComputeStacks. However, if you define a VOLUME in your Dockerfile, but do not define it in ComputeStacks, or have it on a different path than what you have it set to in ComputeStacks, then docker will automatically create that volume on the node at boot.

The ComputeStacks solution to this problem is to simply detect these automatically created volumes, and import their configuration as if they were defined in ComputeStacks.

The problem arises when a user creates a VOLUME directive for the parent directory of the ComputeStack’s defined volume. Here is a common example we see:

In your Dockerfile

VOLUME /var/www

In ComputeStacks

/var/www/html

This scenario will cause ComputeStacks to skip importing the volume created by the Dockerfile because the child path, /var/www/html, is already in use. Docker has no problem creating nested volumes, however that is not supported by ComputeStacks.

If you were inspect the container, you will see this under Mounts:

"Mounts": [
    {
        "Type": "volume",
        "Name": "970502a6-e556-1833-b467-b5e58311d742",
        "Source": "/var/lib/docker/volumes/970502a6-e556-1833-b467-b5e58311d742/_data",
        "Destination": "/var/www/html",
        "Driver": "local",
        "Mode": "z",
        "RW": true,
        "Propagation": ""
    },
    {
        "Type": "volume",
        "Name": "b514aa84cbc85b8fee93ea699eb05b72d231df0c82ad1a8173059990cfa5980c",
        "Source": "/var/lib/docker/volumes/b514aa84cbc85b8fee93ea699eb05b72d231df0c82ad1a8173059990cfa5980c/_data",
        "Destination": "/var/www",
        "Driver": "local",
        "Mode": "",
        "RW": true,
        "Propagation": ""
    }
],

Notice the UUID Name on the first volume, this is created by ComputeStacks. The second volume has the standard docker hexadecimal name format.

Solution

The recommended resolution for this is to either update your Dockerfile to match the volume path defined in ComputeStacks, or remove the VOLUME directive altogether. Once you build and push your updated image, issue a rebuild in ComputeStacks to resolve this issue.

For administrators that are attempting to resolve this without modifying their customer’s custom images, you have an option to update ComputeStacks instead. However, this process is a bit more involved because you will need to modify the files inside of the volume to match the new path.

Modify ComputeStacks Volume Path

First, stop the container in ComputeStacks

If you try to just stop it on the host, CS will restart it again!

Edit the volume in the ComputeStacks Console

Enter the ComputeStacks console (cstacks console on the controller)

Change the UUID and /var/www to match your example. /var/www is the path defined in the Dockerfile.

Volume.find_by(name: '570502a6-e556-4833-b467-a2e58311d742').update container_path: "/var/www"

Migrate Volume Data

If you’re using NFS shared storage, update the data directly on the NFS server, and adjust paths accordingly.

On the host that holds the container and volume, find the path on disk by inspect the container and update the structure of the CS-created volume with the correct folder structure and data.

In this example, ComputeStacks is mounted at /var/www/html, but the Dockerfile is expecting /var/www. So I would do something like:

Notice that i’m intentionally going above the volume folder _data

cd /var/lib/docker/volumes/570502a6-e556-4833-b467-a2e58311d742
mkdir html
mv _data/* html/
mv html _data/

Now you need to ensure permissions are correct – you can do that by using ls -la, but in this example we will use:

chown dockremap:dockremap /var/lib/docker/volumes/570502a6-e556-4833-b467-a2e58311d742/_data/html
chmod 755 /var/lib/docker/volumes/570502a6-e556-4833-b467-a2e58311d742/_data/html

Rebuild the Container

Back in ComputeStacks, issue a container rebuild.

Verify

Check your work by looking at the container logs to see if there are any errors, and to once again inspect the container on the node. You should only see ComputeStack created volumes.