Link Health

Verifies that all expected fabric and host-side links are operationally up, that no links are flapping, and that the EventFSM correctly classifies link-state transitions. Distinct from PHY-error validation, this section is about operational state — is the link up — rather than physical-layer error rates.

Persona Use Cases

Validation Procedure

Step 1: Confirm NIC Link Health on AI Cluster Health

1. In the user-tenant, open AI Cluster Health.

2. Inspect Health Issues (Summary Tiles):

   • NIC Link Flaps/Downs: should be 0.

   • Persistent NIC Link Flaps: should be 0.

   • All Link Downs: should be 0.

   • All NIC Link Downs: should be 0.

3. Each tile isolates a specific counter; drill into any non-zero tile to see the affected devices.

Step 2: Validate Per-tier Link State on Professor-X

1. Open the Professor-X Dashboard.

2. Inspect Link Status & Physical Errors > Link Status (Multi-Tier). The widget tracks oper_state_downs for:

   • Leaf-to-NIC

   • Leaf-to-Spine

   • Spine-to-Super Spine

3. All three should report zero operationally-down interfaces during the idle window.

4. Inspect FSM Link Down Events. The Fabric Manager-specific link-down counter should be quiet.

Step 3: Validate Traffic and Congestion Link Heatmaps

1. Open the Traffic and Congestion dashboard.

2. Inspect Link & Interface Health > Link Status Heatmaps for Leaf to NIC, Leaf to Spine, and Spine to Super Spine.

3. Each heatmap shows operationally-down interfaces by oper_state. All three heatmaps should be empty.

Step 4: Cross-check the Log Errors Heatmap

1. On AI Cluster Health > Log Errors, confirm the Network Link Disconnections or Instability Collective Learning category is not active across hostnames.

Step 5: (Optional) Controlled Link Toggle to Validate the EventFSM

If the change-control process allows, with the network-ops owner present:

1. Toggle a known link by administratively shutting and re-opening it on the switch.

2. Observe the FSM Link Down Events panel on Professor-X Dashboard: the event should appear within the FSM polling window.

3. Confirm the link is restored on the relevant tier's heatmap.

4. Document the test in the ORR baseline archive.

Reference Dashboards and Sections

• AI Cluster Health: Health Issues, Log Errors.

• Professor-X Dashboard: Link Status & Physical Errors.

• Traffic and Congestion: Link & Interface Health.

Pass Criteria

• [ ] All AI Cluster Health link-related summary tiles read 0 during the idle window.

• [ ] Professor-X Dashboard Link Status (Multi-Tier) reports zero oper_state_downs for Leaf-to-NIC, Leaf-to-Spine, and Spine-to-Super Spine.

• [ ] FSM Link Down Events on Professor-X Dashboard is quiet.

• [ ] All three Traffic and Congestion Link Status Heatmaps are empty.

• [ ] Network Link Disconnections or Instability Collective Learning category is not active across hostnames.

• [ ] (If performed) Controlled link-toggle test produced an FSM event within the polling window and restored cleanly.

• [ ] Idle-baseline screenshots archived.

Common Pitfalls

• Link in MAINTENANCE state appears down. Devices with deviceAdminState = MAINTENANCE will register as oper-down on some panels. Filter by admin state when interpreting heatmaps; "down" is only a problem when admin state is up.

• Bootstrap-time flaps after a recent install. If the cluster was just powered on, expect transient flap counts in the early part of the validation window. Re-scope the time filter to a settled window before sign-off.

• Single-tier mismatch. Leaf-to-NIC links flapping while Leaf-to-Spine links are clean usually points to host-side NIC/cable issues rather than fabric-side. The NIC Link Flaps/Downs summary on AI Cluster Health is specifically broken out for this reason.

• Hidden links in tenant scoping. Some preconfigured views filter by role (for example, or server, not server). A link reported clean by one panel and dirty by another may simply be scoped out of one of them: check the view's filter pane.

Last updated: October 14, 2026