Nomad Series - Installation

Series Articles

• Nomad Series Articles

Introduction to Nomad

Starting a new series! I’ve recently finished setting up my home lab environment, so I can finally dive into the real topics.

First up is the HashiCorp Nomad series — welcome aboard.

I briefly introduced Nomad in Large-Scale IoT Edge Container Cluster Management Architectures - 2 - HashiCorp Solution Nomad. Here’s a quick recap:

Nomad: A simple and flexible scheduler and orchestrator (primarily for containers, but not limited to them) that deploys and manages containerized and non-containerized applications at scale, both on-premises and in the cloud.

Nomad enables developers to deploy applications using declarative infrastructure as code. It uses bin packing to efficiently schedule jobs and optimize resource utilization.

Nomad differentiates itself through its simplicity, flexibility, scalability, and high performance. The synergy and integration points with HashiCorp Terraform, Consul, and Vault make it particularly well-suited for easy integration into an organization’s existing workflows, minimizing time-to-market for critical initiatives.

Key Features of Nomad

• Unlike Consul and Kubernetes, Nomad divides infrastructure into regions served by a single Nomad server cluster, but can manage multiple datacenters or availability zones.
• The latency from Nomad clients to their servers can exceed 100 milliseconds. This allows a set of Nomad servers to serve all clients that may be geographically distributed across a continent or even the world, with a single “global” region and multiple datacenters.

Installing Nomad

Nomad is available as a pre-compiled binary and also as a package for several operating systems. Here we’ll install it via a package manager.

Using Ubuntu/Debian as an example:

Prerequisites

• Root privileges
• Fully functional iptables
• Docker installed

│ 🐾Warning
│
│ Note that if you are running Nomad on Linux, you will need to run the client agent as root (or with sudo) for cpuset accounting and network namespaces to work correctly.

Installing via APT

First, install the required packages:

sudo apt-get update && \
  sudo apt-get install wget gpg coreutils

Next, add the HashiCorp GPG key:

wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg

Add the official HashiCorp Linux repository:

echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list

Update and install:

sudo apt-get update && sudo apt-get install nomad

When installed via APT, Nomad is automatically configured as a systemd service. You can enable it to start on boot:

sudo systemctl enable nomad

Post-Installation Steps - Installing CNI Plugins

These steps are considered optional but are helpful for running Nomad and leveraging other Nomad features.

When using bridge networking mode, Nomad uses CNI plugins to configure network namespaces. All Linux Nomad client nodes that use network namespaces must have CNI plugins installed.

The following commands install the CNI reference plugins:

curl -L -o cni-plugins.tgz "https://github.com/containernetworking/plugins/releases/download/v1.3.0/cni-plugins-linux-$( [ $(uname -m) = aarch64 ] && echo arm64 || echo amd64)"-v1.3.0.tgz && \
  sudo mkdir -p /opt/cni/bin && \
  sudo tar -C /opt/cni/bin -xzf cni-plugins.tgz

Ensure your Linux distribution is configured to allow container traffic over bridge networks to be routed through iptables. These tunables can be set as follows:

echo 1 | sudo tee /proc/sys/net/bridge/bridge-nf-call-arptables && \
  echo 1 | sudo tee /proc/sys/net/bridge/bridge-nf-call-ip6tables && \
  echo 1 | sudo tee /proc/sys/net/bridge/bridge-nf-call-iptables

To persist these settings across client node reboots, add a file with the following content to /etc/sysctl.d/bridge.conf:

net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

Verifying the Installation

To verify that Nomad was installed correctly, try the nomad command:

$ nomad version
Nomad v1.6.1
BuildDate 2023-07-21T13:49:42Z
Revision 515895c7690cdc72278018dc5dc58aca41204ccc

Configuration

│ 📝Notes
│
│ We do not start Nomad in dev mode.

Nomad is divided into Nomad Server and Nomad Client. This installation uses:

• 1 Nomad Server (server only, no client running on the same node)
• Multiple Nomad Clients

Some configuration settings are common to both server and client Nomad agents, while others must only be present on one or the other.

Common Nomad Configuration

Create the configuration file at /etc/nomad.d/nomad.hcl:

sudo mkdir --parents /etc/nomad.d
sudo chmod 700 /etc/nomad.d
sudo touch /etc/nomad.d/nomad.hcl

Add this configuration to the nomad.hcl configuration file:

│ 🐾Warning:
│
│ Replace the datacenter parameter value with the identifier of the datacenter where you are deploying the Nomad cluster, as appropriate for your environment.

datacenter = "dc1"
data_dir = "/opt/nomad"

• datacenter - The datacenter in which the agent is running
• data_dir - The data directory where the agent stores state

Server Configuration

Create the configuration file at /etc/nomad.d/server.hcl:

sudo touch /etc/nomad.d/server.hcl

Add this configuration to the server.hcl configuration file:

│ 🐾Warning:
│
│ Replace the bootstrap_expect value with the number of Nomad servers you are deploying; three to five is recommended.
│ In my case, this is a home lab environment, so 1 is sufficient.

server {
  enabled = true
  bootstrap_expect = 1
}

This server stanza contains the following parameters:

• enabled - Specifies whether this agent should run in server mode. All other server options depend on this value being set.
• bootstrap_expect - The expected number of servers in the cluster. Either do not provide this value, or it must match the number of servers in the cluster.

Client Configuration

Create the configuration file at /etc/nomad.d/client.hcl:

sudo touch /etc/nomad.d/client.hcl

Add this configuration to the client.hcl configuration file:

client {
  enabled = true
  servers = ["192.168.2.1"]
}

This client stanza contains the following parameters:

• enabled - Specifies whether this agent should run in client mode. All other client options depend on this value being set.
• servers - List of Nomad Server addresses

│ 🐾Warning:
│
│ This configuration has not been security-hardened. Neither ACL nor TLS has been configured.

Starting Nomad

sudo systemctl start nomad
sudo systemctl status nomad

❯ sudo systemctl status nomad
● nomad.service - Nomad
   Loaded: loaded (/usr/lib/systemd/system/nomad.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2023-08-26 11:15:11 CST; 11h ago
     Docs: https://nomadproject.io/docs/
 Main PID: 1648 (nomad)
    Tasks: 13
   CGroup: /system.slice/nomad.service
           └─1648 /usr/bin/nomad agent -config /etc/nomad.d

At this point, both the Nomad Server and Client are up and running. You can access the UI at http://<server_ip>:4646. Here’s what it looks like:

🎉🎉🎉

📚️References

• Installing Nomad for Production | Nomad | HashiCorp Developer