Networking Minerva User Guide
Networking Minerva User Guide
Breadcrumbs

Layer 3

Verifies that BGP sessions are established between every expected peer pair, and that Adaptive Routing is operating without hardware resource exhaustion. The current preconfigured dashboard set does not include a dedicated Layer 3/BGP dashboard: ORR validation is performed via Fabric Validation and Log Insights. The Performance Degradation (Adaptive Routing Hardware Resource Exhaustion) section describes the kind of issue this validation is meant to catch.

Persona Use Cases

Persona

Primary Use Case

Network Engineers

Confirms every BGP peering between leaf, spine, and super-spine is in the Established state.

System Administrators / SREs

Validates the absence of Adaptive Routing "No More Resources" notices in switch logs that signal hardware key-value DB or neighbor-table exhaustion.

Validation Procedure

Step 1: Validate BGP Session State via Fabric Validation

  1. In the user-tenant, open Fabric Validation.

  2. In Recent Validation Events, confirm that no BGP Not Established violations appear in the validation window.

  3. On the BGP Not Established chart under Validation Classes, confirm that no device shows BGP failures across the validation window.

Step 2: Spot-check via Topology

  1. Open the Topology page.

  2. Pick a representative leaf switch and confirm its BGP neighbors and link adjacencies match the design.

  3. Repeat for one spine and one super-spine.

Step 3: Check for Adaptive Routing Hardware Exhaustion

The Adaptive Routing Hardware Resource Exhaustion case in Performance Degradation surfaces as a NOTICE ROUTER: Failed to add entry to HW module. err: No More Resources log message on Cumulus switches. The ORR should confirm that this signal is absent.

  1. Open Log Insights (in the user-tenant).

  2. Search syslog content for No More Resources, AdaptiveRoutingNotice, or the Cumulus ROUTER application.

  3. Confirm the result set is empty across the idle window.

  4. Also inspect the Collective Learning category AdaptiveRoutingNotice if it exists in your deployment's Collective Learning library. This category is the supervised-learning hook that recognizes these messages programmatically.

Step 4: Confirm Switch Sensor and Routing-process Counters Are Clean

  1. On AI Cluster Health > Log Errors, scan for any router-process or routing-protocol categories in the Collective Learning heatmap.

  2. Confirm that no kernel-level routing alarms (for example, FIB rejection events) appear.

Reference Dashboards and Sections

  • Fabric Validation: BGP Not Established violations and the BGP Not Established chart.

  • Topology page: spot-check leaf/spine/super-spine neighbor relationships.

  • Log Insights (in the user-tenant): for No More Resources and Adaptive Routing notices.

  • AI Cluster Health > Log Errors: Collective Learning categories.

  • Performance Degradation: the canonical example of why this validation matters.

Pass Criteria

  • [ ] No BGP Not Established violations appear in Fabric Validation during the validation window.

  • [ ] No No More Resources or AdaptiveRoutingNotice log messages are present in Log Insights.

  • [ ] The AdaptiveRoutingNotice Collective Learning category (if present) shows no hits.

  • [ ] The spot-checked leaf, spine, and super-spine switches in Topology have the expected neighbor relationships.

  • [ ] Idle-baseline notes archived (count of expected BGP sessions; explicit "no AR exhaustion notices" sign-off).

Common Pitfalls

  • BGP sessions on switches without a role aren't checked. Fabric Validation evaluates only switches that have a fabric role, so assign roles to every leaf, spine, and super-spine before relying on BGP results.

  • A BGP session "missing" is harder to see than a BGP session "down." A peer that never came up at all produces no events. The validation step above relies on counting expected sessions; if the design specifies 96 sessions and the switches report 95, the missing one is the issue.

  • Adaptive Routing exhaustion is subtle. The No More Resources message can appear long before user-visible job impact: the use case in Performance Degradation took one to two weeks to hit production triage without this signal. ORR is the right time to confirm baseline absence.

  • No dedicated dashboard means weaker pass criteria. Until a Layer 3 dashboard ships, this page's pass criteria depend on the operator knowing what to look for. Capture the search queries used in Log Insights in the baseline archive so subsequent operators can re-run the same checks.

Last updated: