Nomad Series - Mounting Storage Volumes in Nomad

Series Articles

• Nomad Series Articles

Overview

Obviously, if Nomad is going to run stateful workloads, mounting storage volumes is an essential feature.

Nomad allows users to mount persistent data from local or remote storage volumes into task environments in several ways:

• Container Storage Interface (CSI) plugins
• Nomad Host Volume support
• Docker Volume drivers

Without CSI installed by default, the primary method used is Nomad Host Volumes.

Nomad Host Volumes allow any directory on a Nomad client to be mounted into an allocation. These directories can be simple directories on the client machine, but they can also be mounted filesystems such as NFS or GlusterFS. These mounts can then be connected to individual tasks within a task group.

Mounting the Tailscale Socket

In a later article, Traefik needs to communicate with Tailscale via its socket to obtain certificates. We can mount the socket into the Traefik container using a Nomad Host Volume (read-only).

Introduction to Nomad Host Volumes

Nomad Host Volumes can manage storage for stateful workloads running within a Nomad cluster.

Nomad Host Volumes provide a workload-agnostic way to specify resources and can be used with Nomad drivers such as exec, java, and docker.

Steps to Use Nomad Host Volumes

Create the Host Directory

On a Nomad client node in the cluster, create a directory for persisting MySQL data. For this example, let’s create the directory /opt/mysql/data:

sudo mkdir -p /opt/mysql/data

Configure the Nomad Client

Edit the Nomad configuration on the corresponding Nomad client to create a host volume.

Add a host_volume block to the client block of the Nomad configuration:

host_volume "mysql" {
  path      = "/opt/mysql/data"
  read_only = false
}

Save the file, then restart the Nomad service on this client to activate the host volume. On the client, you can verify that the host volume is configured using the nomad node status command as shown below:

$ nomad node status -short -self
ID           = 12937fa7
Name         = ip-172-31-15-65
Class        = <none>
DC           = dc1
Drain        = false
Eligibility  = eligible
Status       = ready
Host Volumes = mysql
Drivers      = docker,exec,java,mock_driver,raw_exec,rkt
...

Using Nomad Host Volumes in a Job

Taking a MySQL Job mysql.nomad.hcl as an example:

job "mysql-server" {
  type        = "service"

  group "mysql-server" {
    count = 1

    volume "mysql" {
      type      = "host"
      read_only = false
      source    = "mysql"
    }

    task "mysql-server" {
      driver = "docker"

      volume_mount {
        volume      = "mysql"
        destination = "/var/lib/mysql"
        read_only   = false
      }

      env = {
        "MYSQL_ROOT_PASSWORD" = "password"
      }

      config {
        image = "hashicorp/mysql-portworx-demo:latest"
        ports = ["db"]
      }
    }
    network {
      port "db" {
        static = 3306
      }
    }
  }
}

Here’s a detailed explanation:

• volume “mysql” {: Specifies the volume to use, similar to a PV in Kubernetes. Specifically:
  • type = “host”: Nomad Host Volume type
  • read_only = false: Not read-only
  • source = “mysql”: The source is the host_volume “mysql” configured on the Nomad Client
• volume_mount {: The volume_mount block in the Docker Driver, specifying the mount path inside the container, similar to a PVC in Kubernetes:
  • volume = “mysql”: References the mysql volume defined above
  • destination = “/var/lib/mysql”: Mounts to the /var/lib/mysql directory inside the container
  • read_only = false: The read_only setting of the volume_mount block

Start the job to use the host volume:

nomad run mysql.nomad.hcl

Done 🎉🎉🎉

Summary

In this article, we configured a host volume on a Nomad client using a local directory. We created a job that mounts this volume into a Docker MySQL container, enabling data to be written to the host volume. This lays the foundation for the upcoming Nomad + Traefik + Tailscale setup.

📚️ References

• Stateful Workloads with Nomad Host Volumes | Nomad | HashiCorp Developer