Troubleshoot NVLink Services with Admin Endpoints

Use the admin endpoints of the NVIDIA Minerva NVLink services to check that a service is running, change its log level, and capture its configuration and metrics, for example for a support case. You need access to Docker on a node of the Platform Stack's Docker Swarm cluster.

Admin Endpoints for NMX-M Services

Each NVIDIA Minerva NVLink (NMX-M) microservice listens on a second HTTP address, admin-addr, used for health checks, Prometheus metrics, runtime logging, and configuration inspection during troubleshooting. This interface is separate from the NVIDIA Minerva NVLink REST API.

Use these endpoints when troubleshooting a specific NMX-M service on the NVIDIA Minerva cluster, typically under the guidance of NVIDIA support, or when you need evidence for a support case:

• Raise log verbosity temporarily (/log-level) without restarting the service

• Confirm the process admin listener is up (/live)

• Capture the service's effective config for a support case (/config_dump)

The admin endpoints are not exposed outside the platform's internal network, and they require no authentication. Restrict access to trusted cluster operators.

Admin Ports

Each service listens on its own admin port:

Each service listens on its own port inside its own container; always use the service name together with its port.

Admin Endpoints

Example Workflow

Run the commands from any node of the Platform Stack's Docker Swarm cluster, as a user with access to Docker (for example, with sudo). Replace <service> and <admin-port> with a service and port from the table above, for example nvl-service-mgr and 9084.

1. Confirm the admin listener is up. The output is OK:

   docker exec $(docker ps -q -f name=backend_utility) \
     curl -sS http://<service>:<admin-port>/live
   

2. Check the current log level. Example output: {"level":"info"}

   docker exec $(docker ps -q -f name=backend_utility) \
     curl -sS http://<service>:<admin-port>/log-level
   

3. Set the log level to debug:

   docker exec $(docker ps -q -f name=backend_utility) \
     curl -sS -X PUT -H "Content-Type: application/json" \
     -d '{"level":"debug"}' http://<service>:<admin-port>/log-level
   

4. Reproduce the issue, then restore the log level to info:

   docker exec $(docker ps -q -f name=backend_utility) \
     curl -sS -X PUT -H "Content-Type: application/json" \
     -d '{"level":"info"}' http://<service>:<admin-port>/log-level
   

5. Capture the service configuration:

   docker exec $(docker ps -q -f name=backend_utility) \
     curl -sS http://<service>:<admin-port>/config_dump
   

6. Check the Prometheus metrics:

   docker exec $(docker ps -q -f name=backend_utility) \
     curl -sS http://<service>:<admin-port>/metrics
   

Last updated: October 29, 2026