Prepare Hosts for Docker Swarm

These requirements are for Docker Swarm installations. For Kubernetes, see Kubernetes Requirements.

Unless a section says otherwise, these requirements apply to every NVIDIA Minerva node, including collector nodes.

The installation commands on this page require sudo. NVIDIA recommends creating a dedicated non-root account with passwordless sudo access and Docker group membership.

Supported Operating Systems

• Ubuntu 24.04 LTS (x86-64 or ARM64)

Required Packages

Install the following on every node before running the installer.

System Packages

sudo apt-get update
sudo apt-get install -y \
  docker.io python3 python3-yaml ca-certificates openssl \
  logrotate rsyslog cron curl rsync git systemd-coredump auditd \
  tar gzip unzip

AWS CLI

Install AWS CLI v2. Download the package for the host's architecture.

On x86-64 (amd64) hosts:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

On ARM64 (aarch64) hosts:

curl "https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

To check the host's architecture, run uname -m. It returns x86_64 or aarch64.

Docker Engine

Docker version 20.10.23 or later is required. Version 27.5.0 is recommended. The installer verifies the Docker Engine version on every node and fails if any node reports lower than 20.10.23. If the distribution's default docker.io package is older than this, install Docker from the official Docker repository instead, or upgrade it before proceeding. Verify with:

docker version --format '{{.Server.Version}}'

Start and enable the daemon:

sudo systemctl enable --now docker

Host Configuration

Disk Size

Minimum recommended values:

• At least 20 GB free in $HOME

• At least 50 GB for /var/lib/docker

• ~/opt sized for the selected template and retention policy (platform nodes)

For larger deployments, use the sizing recommended by NVIDIA support.

Data Directory Mount Options (Platform Nodes Only)

If the data directory (/data or equivalent) is on its own XFS filesystem, mount it with:

noatime,logbsize=256k,largeio,inode64,nofail

These options are XFS-specific; a non-XFS filesystem should be reformatted as XFS rather than substituted with an equivalent, as no reliable ext4 equivalent exists for logbsize, largeio, or inode64.

Verify both the live mount and the persisted fstab entry — the two can drift independently:

findmnt -T /data
grep -v '^#' /etc/fstab | grep /data

nofail is a mount(8)/boot-time directive and will not appear in findmnt output even when correctly configured — check it in /etc/fstab only.

Swap

Disable swap on every node:

sudo swapoff -a

Remove or comment out any swap entry in /etc/fstab so it does not re-enable on reboot. Verify:

cat /proc/swaps

Time Synchronization

Synchronize the system clock on every node with NTP. TLS connections between collectors and the platform fail if their clocks are wrong.

Verify:

timedatectl | grep -E 'synchronized|NTP service'

The output must include System clock synchronized: yes.

systemd-coredump Configuration

Route core dumps through systemd-coredump. These commands remove any hand-written core_pattern override and restore the package default:

sudo rm -f /etc/sysctl.d/50-coredump.conf
sudo apt-get install -y systemd-coredump
sudo sysctl --system
cat /proc/sys/kernel/core_pattern

The output must begin with | and contain systemd-coredump.

Core dumps land in /var/lib/systemd/coredump on the root volume, and the default limits allow up to 32 GB per dump. A large heap can produce a dump big enough to fill the root filesystem. Set explicit limits:

sudo tee /etc/systemd/coredump.conf > /dev/null <<'EOF'
[Coredump]
Compress=yes
ProcessSizeMax=8G
ExternalSizeMax=8G
MaxUse=10G
KeepFree=20G
EOF
sudo systemctl daemon-reload

KeepFree is the important setting — systemd refuses to write a dump if free space would fall below it, protecting the root filesystem. A process larger than ProcessSizeMax is skipped entirely, not truncated; raise the value only if the root volume has room for it. Verify:

coredumpctl list --no-pager

auditd Configuration

NVIDIA Minerva records execve system calls for audit trails. The installer warns if auditd is not configured.

Add the following audit rules:

sudo tee -a /etc/audit/rules.d/audit.rules > /dev/null << 'EOF'
-a exit,always -F arch=b32 -S execve -k auditcmd
-a exit,always -F arch=b64 -S execve -k auditcmd
EOF

sudo service auditd restart

Verify the rules are loaded:

sudo auditctl -l | grep auditcmd

sysctl I/O Tuning

Apply the following on every node:

sudo tee /etc/sysctl.d/70-minerva-io.conf > /dev/null << 'EOF'
vm.dirty_background_bytes = 268435456
vm.dirty_bytes            = 2147483648
vm.vfs_cache_pressure     = 50
vm.max_map_count          = 1048576
EOF
sudo sysctl --system

Verify:

sysctl vm.dirty_background_bytes vm.dirty_bytes vm.vfs_cache_pressure vm.max_map_count

Use sysctl --system, not sysctl -p against an individual file — the platform also ships a separate sysctl file using ratio-based dirty-page settings (vm.dirty_ratio/vm.dirty_background_ratio), which are mutually exclusive with the byte-based settings above. sysctl --system applies all files in the correct order and avoids silently zeroing out either set.

Configure Logrotate

Cap the systemd journal so it cannot independently fill the disk:

sudo mkdir -p /etc/systemd/journald.conf.d
sudo tee /etc/systemd/journald.conf.d/limits.conf >/dev/null <<'EOF'
[Journal]
SystemMaxUse=1G
SystemMaxFileSize=100M
SystemKeepFree=2G
EOF
sudo systemctl restart systemd-journald

Verify:

journalctl --disk-usage

NVIDIA Minerva generates high-volume syslog traffic. Ensure hourly log rotation is configured for /var/log/syslog to prevent disk exhaustion. Do this on every node.

Create /etc/cron.hourly/logrotate.syslog if it does not exist:

sudo tee /etc/cron.hourly/logrotate.syslog >/dev/null << 'EOF'
#!/bin/sh
/usr/sbin/logrotate /etc/logrotate.d/rsyslog
EOF

sudo chmod 755 /etc/cron.hourly/logrotate.syslog

Add a maxsize directive to /etc/logrotate.d/rsyslog to cap individual log file size (for example, maxsize 500M). Before adding the directive, check for an existing maxsize setting to avoid duplicate entries.

If you back up the existing rsyslog config, place the backup outside /etc/logrotate.d/. Files in that directory are processed by logrotate, and a misplaced backup will cause duplicate log rotation entries.

User Setup

Add the user that will run svm to the docker group:

sudo usermod -a -G docker <user>

Log out and back in for the change to take effect.

The installation user must also be allowed to use crontab. If it isn't, the installer still completes, but prints error: crontab failed (/etc/cron.allow: Permission denied). In that case, ask an administrator to allow the user, and then run ./svm install-cron from ~/minerva.

Copy the NVIDIA Minerva Installer to the Home Directory

This guide assumes the NVIDIA Minerva installer is located in the home directory of the installation user.

Manager SSH Access

Some svm operations use SSH/SCP from the manager to worker nodes. For multi-node installations, the manager installation user must be able to SSH to each worker without a password.

From the manager, create an SSH key if the installation user does not already have one:

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519

Copy the manager public key to each worker:

ssh-copy-id <worker-user>@<worker-host>

If ssh-copy-id is not available, append the manager public key to the worker user's ~/.ssh/authorized_keys.

Verify access from the manager:

ssh <worker-user>@<worker-host> hostname

Network Requirements

Docker Subnet

NVIDIA Minerva uses 192.168.0.0/16 for internal Docker overlay networking by default. This subnet must not overlap with any on-premises subnets reachable by monitored devices — overlap will break gNMI collection. The subnet must be at least /24 (256 addresses).

Hosts with Multiple Interfaces or Addresses

If a host has more than one network interface, or more than one IP address on the same interface, specify the interface or address Docker Swarm should advertise. Add the following to /etc/docker/daemon.json on every node, including collector nodes:

{
  "swarm-default-advertise-addr": "<interface-name-or-ip>"
}

Restart Docker after editing. Without this setting, Swarm creation fails on such a host with: could not choose an IP address to advertise since this system has multiple addresses on interface

DNS Options

Check the DNS options on every node:

grep -E '^options' /etc/resolv.conf

If the output shows ndots:1, add this line inside the JSON object in /etc/docker/daemon.json (add a comma if other entries follow it):

"dns-opts": ["ndots:0"]

If the file also sets swarm-default-advertise-addr (see Hosts with Multiple Interfaces or Addresses), both settings go in the same object:

{
  "swarm-default-advertise-addr": "<interface-name-or-ip>",
  "dns-opts": ["ndots:0"]
}

Then restart Docker:

sudo systemctl restart docker

HTTP Access

Every node needs outbound HTTPS access to:

• The configured container registry, unless using a bundled-image installer

• S3 or Azure object storage, if used

• Any configured support-bundle upload destination

Container Registry Authentication (NMC only)

If the provided installer is not the full installer (installer.minerva.all.<BUILD>), you must log in to the container registry on all nodes with your nvcr.io credentials.

docker login nvcr.io

HTTP Proxy

If outbound access requires a proxy, configure it on every node before running the installer. Edit /etc/environment:

http_proxy=http://PROXY_HOSTNAME:PROXY_PORT
https_proxy=http://PROXY_HOSTNAME:PROXY_PORT
no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com"
HTTP_PROXY=http://PROXY_HOSTNAME:PROXY_PORT
HTTPS_PROXY=http://PROXY_HOSTNAME:PROXY_PORT
NO_PROXY="localhost,127.0.0.1,localaddress,.localdomain.com"

Log out and back in after saving.

Multi-Node Swarm Port Requirements

In a multi-node deployment, open the following ports.

All Nodes (Managers and Workers)

Manager Nodes Only

Node Running the Ingest Task

Hybrid Deployments: Collector to Platform Ports

In a hybrid deployment, open the ports listed in Inbound to Platform Nodes (from Collectors) in Collector Installation. Make sure 14317 and 24317 are open: collectors send data-plane metrics (14317) and their own health metrics and logs (24317) over them.

Loopback Routing for Self-Addressed Traffic

Some host-hardening firewall baselines include an anti-spoof rule that rejects traffic addressed to 127.0.0.0/8 when it arrives on a non-loopback interface. On a Docker Swarm node, traffic that round-trips through the ingress routing mesh (for example, a container reaching another local service via its published port) legitimately arrives back at the host over a bridge interface addressed to 127.0.0.1, not over lo.

If an anti-spoof rule of this kind is positioned ahead of the RELATED,ESTABLISHED accept rule in the host's INPUT chain, it silently drops this return traffic. The symptom is a service or health agent that appears to start normally but never completes a connection to a local, loopback-addressed endpoint, with no corresponding error in the host firewall log.

Before deploying, confirm whether such a rule exists:

sudo iptables -S INPUT | grep -i '127.0.0.0/8'

If present and positioned before the established-connection accept rule, add an explicit accept rule ahead of it for established/related traffic arriving on the Docker bridge interface:

sudo iptables -I INPUT 1 -i docker_gwbridge -d 127.0.0.0/8 \
  -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

Persist the change according to the host's standard firewall-persistence mechanism so it survives a reboot.

After you meet these requirements, install the platform (see Platform Installation). For a hybrid deployment, install the collectors after the platform is running (see Collector Installation).

Last updated: October 30, 2026