The following sections guide the operator team (the personas listed under Audience) through the Operational Readiness Review (ORR) of an AI Factory deploying with NVIDIA Minerva. An ORR confirms that the NVIDIA Minerva stack is ingesting telemetry from every configured data source and is ready to observe the AI Factory cluster it monitors. The ORR begins after the NVIDIA Minerva platform and collector stacks are installed (see Get Started) and AI Factory data sources are onboarded, and ends with an initial deployment sign-off indicating that the cluster is ready for workloads.
Outcomes of a Successful ORR
By the end of the ORR, the operator team produces:
-
A Pass Criteria checklist for each of the ten validation areas below.
-
An archived idle baseline: screenshots of the AI Cluster Health, GPU Summary by Node, NVLink Health Dashboard, Professor-X Dashboard, and Telemetry Health dashboards captured while the cluster is idle. These are the reference points every later "is this normal?" question will be measured against.
-
A gap list: anything not green at sign-off, which carries forward to the next operations phase.
Audience
|
Persona |
Day 0 focus |
|---|---|
|
Infrastructure / DC Ops |
Physical fabric, optics, cabling, power and thermal sensors. |
|
System Administrators / SREs |
Node, NIC, GPU, NVSwitch, kernel-log, and telemetry-pipeline sanity. |
|
Network Engineers |
Leaf/Spine/Super Spine reachability, link state, congestion control, ECMP balance. |
|
AI/ML Engineers / Application Owners |
Job ingestion, scheduler integration, idle GPU baselines for later workload comparison. |
|
Tenant Admins |
Tenant-level inventory totals and per-tenant grouping ( |
Prerequisites
The following must be true before you begin the ORR:
-
NVIDIA Minerva Platform Stack is installed and reachable (see Platform Installation).
-
NVIDIA Minerva Collector Stack is installed on-premises and the gRPC tunnel is established (see Collector Installation).
-
AI Factory data sources are onboarded (see Connect to Data Sources): DOCA Telemetry on hosts, Cumulus OTLP/syslog, NMX-T, Slurm event collectors, and Slurm logs.
-
The operator has UI access to both
default(admin) anduser-tenanttenants and knows which tenant each dashboard set imports into. -
The Reference Configuration Package is available on a workstation that can reach the NVIDIA Minerva UI in the browser (required for the dashboard import in Prepare for Readiness Validation).
If any prerequisite is not met, stop and complete it first: ORR validation against an incomplete platform produces false-negative or false-positive results.
Section Structure
Each subsection that follows uses the same structure:
-
Persona Use Cases: who cares about this validation step and why.
-
Validation Procedure: numbered, imperative steps the operator follows in the UI.
-
Reference Dashboards and Sections: cross-references to the preconfigured dashboards and alerts.
-
Pass Criteria: checklist to complete before advancing to the next area.
-
Common Pitfalls: known failure modes and how they appear in the dashboards.
Validation Areas
Work through the validation areas in the order shown in the table. Complete Pre-Validation Setup first, because every other area assumes its steps are done. Validate Topology next, because the results of every other area depend on correct inventory. NVLink Health cross-checks the GPU Health results, PHY Errors cross-checks NVLink Health, and ECMP and Multiplane Balance relies on the link and plane counts confirmed in Topology. Job Validation is the final validation area, followed by Sign-Off and Handoff.
|
Validation area |
Primary dashboards |
|---|---|
|
n/a — setup |
|
|
AI Cluster Health, Professor-X Dashboard, Traffic and Congestion |
|
|
BGP Sessions, BGP Operations, Fabric Validation, Log Insights |
|
|
AI Cluster Health, Telemetry Health, Job Failure Root Cause Analysis, Trends |
|
|
Performance Dashboard (imported from the Reference Configuration Package), Professor-X Dashboard |
|
|
Job Failure Root Cause Analysis, Telemetry Health, Professor-X Dashboard |
|
|
Aggregate sign-off |
What This Guide Does Not Validate
These are explicitly out of scope for ORR:
-
Anomaly configurations. A subset of potential anomalies are shipped enabled out of the box. Deciding which detectors to turn on, and tuning their thresholds, is a post-ORR activity.
-
Notifications endpoint configurations.
-
Auto Correlation namespace and view binding.
-
Production workload baselines. ORR captures idle baselines only. Active-workload baselines are captured once the cluster is running representative jobs.
Last updated: