This section covers per-GPU and per-node performance tracking. It monitors how individual compute resources interact with the network, ensuring that host-level bottlenecks — such as thermal throttling or PCIe congestion — do not degrade overall cluster throughput.
-
GPU Summary by GPU: Tracks telemetry for individual GPUs, including utilization, clock speeds, and XID errors, to identify specific "straggler" units.
-
GPU Summary by Node: Aggregates health metrics at the node level so you can detect power or cooling issues affecting multiple GPUs within a single server.
Last updated: