The installation commands on this page require sudo. NVIDIA recommends creating a dedicated non-root account with passwordless sudo access and Docker group membership.
Supported Operating Systems
-
Ubuntu 24.04 LTS
Required Packages
Install the following on every node before running the installer.
On Ubuntu:
sudo apt-get update
sudo apt-get install -y \
docker.io python3 python3-yaml ca-certificates openssl \
logrotate rsyslog cron curl rsync git systemd-coredump auditd \
tar gzip unzip
Install AWS CLI v2:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
Docker version 20.10.23 or later is required. Version 27.5.0 is recommended. The installer verifies the Docker Engine version on every node and fails if any node reports lower than 20.10.23. If the distribution's default docker.io package is older than this, install Docker from the official Docker repository instead, or upgrade it before proceeding. Verify with:
docker version --format '{{.Server.Version}}'
Start and enable the daemon:
sudo systemctl enable --now docker
Disk Size
Minimum recommended values:
-
At least 20 GB free in
$HOME -
At least 50 GB for
/var/lib/docker -
~/optsized for the selected template and retention policy
For larger deployments, use the sizing recommended by NVIDIA support.
Data Directory Mount Options
If the data directory (/data or equivalent) is on its own XFS filesystem, mount it with:
noatime,logbsize=256k,largeio,inode64,nofail
These options are XFS-specific; a non-XFS filesystem should be reformatted as XFS rather than substituted with an equivalent, as no reliable ext4 equivalent exists for logbsize, largeio, or inode64.
Verify both the live mount and the persisted fstab entry — the two can drift independently:
findmnt -T /data
grep -v '^#' /etc/fstab | grep /data
nofail is a mount(8)/boot-time directive and will not appear in findmnt output even when correctly configured — check it in /etc/fstab only.
Swap
Disable swap on every node:
sudo swapoff -a
Remove or comment out any swap entry in /etc/fstab so it does not re-enable on reboot. Verify:
cat /proc/swaps
User Setup
Add the user that will run svm to the docker group:
sudo usermod -a -G docker <user>
Log out and back in for the change to take effect.
systemd-coredump Configuration
Route core dumps through systemd-coredump. These commands remove any hand-written core_pattern override and restore the package default:
sudo rm -f /etc/sysctl.d/50-coredump.conf
sudo apt-get install -y systemd-coredump
sudo sysctl --system
cat /proc/sys/kernel/core_pattern
The output must begin with | and contain systemd-coredump.
Core dumps land in /var/lib/systemd/coredump on the root volume, and the default limits allow up to 32 GB per dump. A large heap can produce a dump big enough to fill the root filesystem. Set explicit limits:
sudo tee /etc/systemd/coredump.conf > /dev/null <<'EOF'
[Coredump]
Compress=yes
ProcessSizeMax=8G
ExternalSizeMax=8G
MaxUse=10G
KeepFree=20G
EOF
sudo systemctl daemon-reload
KeepFree is the important setting — systemd refuses to write a dump if free space would fall below it, protecting the root filesystem. A process larger than ProcessSizeMax is skipped entirely, not truncated; raise the value only if the root volume has room for it. Verify:
coredumpctl list --no-pager
auditd Configuration
NVIDIA Minerva records execve system calls for audit trails. The installer warns if auditd is not configured.
Add the following audit rules:
sudo tee -a /etc/audit/rules.d/audit.rules > /dev/null << 'EOF'
-a exit,always -F arch=b32 -S execve -k auditcmd
-a exit,always -F arch=b64 -S execve -k auditcmd
EOF
sudo service auditd restart
Verify the rules are loaded:
sudo auditctl -l | grep auditcmd
sysctl I/O Tuning
Apply the following on every node:
sudo tee /etc/sysctl.d/70-minerva-io.conf > /dev/null << 'EOF'
vm.dirty_background_bytes = 268435456
vm.dirty_bytes = 2147483648
vm.vfs_cache_pressure = 50
vm.max_map_count = 1048576
EOF
sudo sysctl --system
Verify:
sysctl vm.dirty_background_bytes vm.dirty_bytes vm.vfs_cache_pressure vm.max_map_count
Use sysctl --system, not sysctl -p against an individual file — the platform also ships a separate sysctl file using ratio-based dirty-page settings (vm.dirty_ratio/vm.dirty_background_ratio), which are mutually exclusive with the byte-based settings above. sysctl --system applies all files in the correct order and avoids silently zeroing out either set.
Network Requirements
Docker Subnet
NVIDIA Minerva uses 192.168.0.0/16 for internal Docker overlay networking by default. This subnet must not overlap with any on-premises subnets reachable by monitored devices — overlap will break gNMI collection. The subnet must be at least /24 (256 addresses).
Multi-NIC Hosts
On hosts with multiple network interfaces, specify the interface Docker Swarm should advertise by adding the following to /etc/docker/daemon.json on every node:
{
"swarm-default-advertise-addr": "<interface-name-or-ip>"
}
Restart Docker after editing.
DNS Options
Check the DNS options on every node:
grep -E '^options' /etc/resolv.conf
If the output shows ndots:1, add this line inside the JSON object in /etc/docker/daemon.json (add a comma if other entries follow it):
"dns-opts": ["ndots:0"]
If the file also sets swarm-default-advertise-addr (see Multi-NIC Hosts), both settings go in the same object:
{
"swarm-default-advertise-addr": "<interface-name-or-ip>",
"dns-opts": ["ndots:0"]
}
Then restart Docker:
sudo systemctl restart docker
HTTP Access
Every node needs outbound HTTPS access to:
-
The configured container registry, unless using a bundled-image installer
-
S3 or Azure object storage, if used
-
Any configured support-bundle upload destination
Container Registry Authentication (NMC only)
If the provided installer is not the full installer (installer.minerva.all.<BUILD>), you must log in to the container registry on all nodes with your nvcr.io credentials.
docker login nvcr.io
HTTP Proxy
If outbound access requires a proxy, configure it on every node before running the installer. Edit /etc/environment:
http_proxy=http://PROXY_HOSTNAME:PROXY_PORT
https_proxy=http://PROXY_HOSTNAME:PROXY_PORT
no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com"
HTTP_PROXY=http://PROXY_HOSTNAME:PROXY_PORT
HTTPS_PROXY=http://PROXY_HOSTNAME:PROXY_PORT
NO_PROXY="localhost,127.0.0.1,localaddress,.localdomain.com"
Log out and back in after saving.
Multi-Node Swarm Port Requirements
Make sure the following ports are open for communication in a multi-node deployment:
On all nodes (managers + workers)
|
Port |
Protocol |
Purpose |
|---|---|---|
|
7946 |
TCP/UDP |
Swarm node communication |
|
4789 |
UDP |
Docker overlay network (VXLAN) |
|
443 |
TCP |
UI and API (HTTPS) |
|
80 |
TCP |
UI and API (HTTP) |
On manager nodes only
|
Port |
Protocol |
Purpose |
|---|---|---|
|
2377 |
TCP |
Docker Swarm management |
|
33352 |
UDP |
Health-monitor agent report submission (from every platform node, including the manager itself) |
On the node running the ingest task only
|
Port |
Protocol |
Purpose |
|---|---|---|
|
4317 |
TCP |
OTLP gRPC — device telemetry |
|
4318 |
TCP |
OTLP HTTP — device telemetry |
|
4319 |
TCP |
OTLP gRPC — NMX-T (NVLink) ingress |
|
9091 |
TCP |
Prometheus |
|
57402 |
TCP |
gNMI dial-out (WJH) |
|
514 |
UDP |
Syslog |
|
4739 |
UDP |
IPFIX |
Hybrid Deployments: Collector to Platform Ports
In a hybrid deployment, open the ports listed under Inbound to Platform node (from collectors) in Collector Installation. Make sure 14317 and 24317 are open: collectors send data-plane metrics (14317) and their own health metrics and logs (24317) over them.
Loopback Routing for Self-Addressed Traffic
Some host-hardening firewall baselines include an anti-spoof rule that rejects traffic addressed to 127.0.0.0/8 when it arrives on a non-loopback interface. On a Docker Swarm node, traffic that round-trips through the ingress routing mesh (for example, a container reaching another local service via its published port) legitimately arrives back at the host over a bridge interface addressed to 127.0.0.1, not over lo.
If an anti-spoof rule of this kind is positioned ahead of the RELATED,ESTABLISHED accept rule in the host's INPUT chain, it silently drops this return traffic. The symptom is a service or health agent that appears to start normally but never completes a connection to a local, loopback-addressed endpoint, with no corresponding error in the host firewall log.
Before deploying, confirm whether such a rule exists:
sudo iptables -S INPUT | grep -i '127.0.0.0/8'
If present and positioned before the established-connection accept rule, add an explicit accept rule ahead of it for established/related traffic arriving on the Docker bridge interface:
sudo iptables -I INPUT 1 -i docker_gwbridge -d 127.0.0.0/8 \
-m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
Persist the change according to the host's standard firewall-persistence mechanism so it survives a reboot.
Configure Logrotate
Cap the systemd journal so it cannot independently fill the disk:
sudo mkdir -p /etc/systemd/journald.conf.d
sudo tee /etc/systemd/journald.conf.d/limits.conf >/dev/null <<'EOF'
[Journal]
SystemMaxUse=1G
SystemMaxFileSize=100M
SystemKeepFree=2G
EOF
sudo systemctl restart systemd-journald
Verify:
journalctl --disk-usage
NVIDIA Minerva generates high-volume syslog traffic. Ensure hourly log rotation is configured for /var/log/syslog to prevent disk exhaustion. This must be done for all nodes in a multi-node deployment (or just the single node for a single node deployment).
Create /etc/cron.hourly/logrotate.syslog if it does not exist:
sudo tee /etc/cron.hourly/logrotate.syslog >/dev/null << 'EOF'
#!/bin/sh
/usr/sbin/logrotate /etc/logrotate.d/rsyslog
EOF
sudo chmod 755 /etc/cron.hourly/logrotate.syslog
Add a maxsize directive to /etc/logrotate.d/rsyslog to cap individual log file size (for example, maxsize 500M). Before adding the directive, check for an existing maxsize setting to avoid duplicate entries.
If you back up the existing rsyslog config, place the backup outside /etc/logrotate.d/. Files in that directory are processed by logrotate and a misplaced backup will cause duplicate log rotation entries.
Copy the NVIDIA Minerva Installer to the Home Directory
This guide assumes the NVIDIA Minerva installer is located in the home directory of the installation user.
Manager SSH Access
Some svm operations use SSH/SCP from the manager to worker nodes. For multi-node installations, the manager installation user must be able to SSH to each worker without a password.
From the manager, create an SSH key if the installation user does not already have one:
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519
Copy the manager public key to each worker:
ssh-copy-id <worker-user>@<worker-host>
If ssh-copy-id is not available, append the manager public key to the worker user's ~/.ssh/authorized_keys.
Verify access from the manager:
ssh <worker-user>@<worker-host> hostname
Next: Single-node Installation | Multi-Node Installation
Last updated: