Networking Minerva User Guide

AI Factory Networking

The AI Factory Networking dashboard suite provides multi-layer visibility across the entire networking fabric. It translates telemetry data into clear operational signals so you can detect congestion or hardware degradation before they impact cluster performance.

Node and GPU

  • GPU Summary by GPU: Provides a high-resolution look at individual GPU health, focusing on utilization, clock speeds, and thermal telemetry to identify "straggler" GPUs that may be slowing down a job.

  • GPU Summary by Node: Aggregates GPU metrics at the host level so you can see whether specific server chassis are experiencing systemic power or cooling issues affecting all resident GPUs.

  • NVLink Health Dashboard: Monitors the operational status of the high-speed chip-to-chip interconnects, highlighting link-down events or hardware-level signal integrity issues.

  • NVLink Interfaces: Provides detailed port-level statistics for NVLink, including bandwidth utilization and error counters to debug performance degradation.

Spectrum-X

  • Professor-X Dashboard: A specialized view for deep Ethernet fabric analysis, correlating switch telemetry with RDMA performance to support lossless network operation.

  • Traffic and Congestion: Visualizes fabric-wide bottlenecks, tracking Global Pause frames, buffer occupancy, and ECN (Explicit Congestion Notification) marking to pinpoint where data is being queued.

  • Multi-Plane Dashboards: Compares P0–P3 fabric planes across traffic, ECN, drops, and queue pressure to isolate whether a network symptom is plane-specific or fabric-wide.

Last updated: