Networking DOCA SDK Documentation
Networking DOCA SDK Documentation
Breadcrumbs

HBN Service Configuration

To start configuring HBN, log in to the HBN container:

sudo crictl exec -it $(crictl ps | grep hbn | awk '{print $1;}') bash

General Network Configuration

Flat Files Configuration

Add network interfaces and FRR configuration files to HBN. For more more information refer to the Cumulus Linux FRRouting guide.

  • /etc/network/interfaces

  • /etc/frr/frr.conf; /etc/frr/daemons

NVUE Configuration

This section assumes familiarity with NVIDIA user experience (NVUE) Cumulus Linux. The following sections elaborate on HBN-specific aspects of NVUE.

NVUE Service

HBN installs NVUE by default and enables NVUE service at boot.

NVUE REST API

HBN enables the REST API with only localhost access by default. To enable REST API access, refer to Enable REST API Access.

You can run cURL commands from the command line. Use the default HBN username, nvidia, and password which you must be update when enabling the REST API using the HBN preparation script.

To change the default password for the nvidia user or to add additional users for NVUE access, refer to section HBN Service Configuration#HBN NVUE User Credentials.

REST API example:

curl -u 'nvidia:nvidia' --insecure https://<mgmt_ip>:8765/nvue_v1/vrf/default/router/bgp
{
  "configured-neighbors": 2,
  "established-neighbors": 2,
  "router-id": "10.10.10.201"
}

NVUE REST API Management Through CLI

  • To enable the REST API service:

    nv set system api state enabled

  • To disable the REST API service:

    nv set system api state disabled

  • To bind the REST API service to a specific address:

    nv set system api listening-address <localhost|ipv4|ipv6|0.0.0.0>

For information about using the NVUE REST API, refer to the NVUE API documentation.

NVUE REST API in DOCA Platform Framework

For information on DOCA Platform Framework (DPF), refer to DOCA Framework.

This section explains how to configure external access to the HBN NVUE REST API in a DOCA Platform Framework (DPF) environment.

By default, the NVUE REST API is only accessible from within the cluster (i.e., via localhost). To expose the API to external networks, you must configure Kubernetes NodePort and set the appropriate listening address.

Enabling External Access to the NVUE REST API

  1. Configure NodePort in DPUService – Add the following section under spec.serviceConfiguration.helmChart.values in your DPUServiceConfiguration YAML:

    YAML
    service: 
  type: NodePort 
  nodePort: 30765

  2. Set the API listening address – To bind the API to all available HBN IP addresses (for example, for external access), configure the following:

    YAML
    configuration: 
  startupYAMLJ2: |  
    - set: 
        system: 
          api: 
            listening-address: 
              0.0.0.0: {}

  3. Verify API accessibility – Use curl to confirm that the NVUE REST API is reachable:

    Bash
    curl -k -u 'username:password' https://<node-ip>:<nodePort>/nvue_v1/interface/

  4. Check service status – To check the NodePort service status:

    YAML
    kubectl get svc -n <your-namespace>

Example: Combined DPUServiceConfiguration and DPUServiceTemplate

The following is a sample manifest to deploy HBN with the NVUE REST API enabled:

Click to expand...
YAML
--- 
apiVersion: svc.dpu.nvidia.com/v1alpha1 
kind: DPUServiceConfiguration 
metadata: 
  name: doca-hbn 
  namespace: dpf-operator-system 
spec: 
  deploymentServiceName: "doca-hbn" 
  serviceConfiguration: 
    helmChart: 
      values: 
        service: 
          nodePort: 30765 
          type: NodePort 
        configuration: 
          user: 
            create: true 
            password: 
              secretKey: password 
              secretName: hbn-user-password 
            username: nvidia 
          perDPUValuesYAML: | 
            - hostnamePattern: "*" 
          startupYAMLJ2: | 
            - header: 
                model: bluefield 
                nvue-api-version: nvue_v1 
                rev-id: 1.0 
                version: HBN 3.1.0 
            - set: 
                system: 
                  api: 
                    listening-address: 
                      0.0.0.0: {}

Security Considerations

Authentication

  • Always use valid username/password authentication.

  • Use HTTPS to secure all REST API communications.

  • The default credentials are nvidia:nvidia.

  • It is strongly recommended to change the default credentials using the user creation procedure described below.

User Creation/Update Procedure for NVUE REST API

The user should not interact directly with the DPU cluster in DPF.

  1. Create a password secret on the host cluster:

    Bash
    kubectl -n dpf-operator-system create secret generic hbn-user-password \
  --from-literal=password=<new-password>

  2. Propagate the secret to the DPU cluster (label the secret so that it is made available to the DPU):

    Bash
    kubectl -n dpf-operator-system label secret hbn-user-password \
  dpu.nvidia.com/image-pull-secret=""

  3. Update the user configuration in the helm values your service YAML:

    YAML
    configuration: 
  user: 
    create: true 
    username: "your-username" 
    password: 
      secretName: "hbn-user-password" #Must match the secret name from Step 1 
      secretKey: "password"           #Must match the key in the secret 
    # Optional settings 
    shell: "/bin/bash" 
    homeDir: "/home/your-username" 
    additionalGroups: ["group1", "group2"] 
    uid: "1000" 
    gid: "1000"
User Creation Configuration Options

The following options can be used to configure user creation or update in the spec.serviceConfiguration.helmChart.values.configuration.user section.

Option

Required

Description

create

Yes

Enables user creation or update. Default is false.

username

Yes

Username for the new user, or existing user in case of an update

password.secretName

Yes

Name of the Kubernetes secret that stores the password

password.secretKey

Yes

Key within the secret that contains the password value

shell

No

Shell assigned to the user (e.g., /bin/bash)

homeDir

No

Path to the user’s home directory

additionalGroups

No

List of additional groups to assign the user to (e.g., ["group1", "group2"])

uid

No

User ID

gid

No

Group ID

NVUE CLI

For information about using the NVUE CLI, refer to the NVUE CLI documentation

NVUE Startup Configuration File

When the network configuration is saved using NVUE, HBN writes the configuration to the /etc/nvue.d/startup.yaml file.

The startup configuration is applied by following the supervisor daemon at boot time. nvued-startup will appear in an EXITED state after applying the startup configuration:

Bash
# supervisorctl status nvued-startup
nvued-startup                    EXITED    Apr 17 10:04 AM

nv config apply startup applies the yaml configuration saved at /etc/nvue.d/. nv config save saves the running configuration to /etc/nvue.d/startup.yaml.

HBN Configuration Examples

HBN Default Configuration

After a fresh HBN installation, the default /etc/network/interfaces file should contain only the declaration of a loopback interface:

source /etc/network/interfaces.d/*.intf
 
auto lo
iface lo inet loopback

FRR configuration files are also present in /etc/frr/ but configurations are not enabled.

Layer-3 Routing

Next-hop IDs and Next-hop Groups

In HBN, route programming utilizes next-hop IDs to specify a route's next-hop. For routes with multiple paths, next-hop groups are used to identify equal-cost multi-path (ECMP) next-hop sets. This grouping enables the system to efficiently process and modify route and next-hop objects, improving scalability, network convergence, and performance. You may notice changes in the display of route information in the output of kernel operational commands.

Native Routing with BGP and ECMP

HBN supports unicast routing with BGP and ECMP for IPv4 and IPv6 traffic. ECMP is achieved by distributing traffic using hash calculation based on the source IP, destination IP, and protocol type of the IP header.

For TCP and UDP packets, the source and destination ports are also included.

ECMP Example

ECMP is implemented any time routes have multiple paths over uplinks or host ports. For example, 20.20.20.0/24 has 2 paths using both uplinks, so a path is selected based on a hash of the IP headers.

Bash
# ip route show 20.20.20.0/24
20.20.20.0/24 nhid 106 proto bgp metric 20
# ip nexthop show group id 106
id 106 group 105/107 proto zebra
# ip nexthop show id 105     
id 105 via fe80::202:ff:fe00:7 dev p0_if scope link proto zebra
# ip nexthop show id 107
id 107 via fe80::202:ff:fe00:c dev p1_if scope link proto zebra

HBN supports up to 64 paths for ECMP

Sample NVUE Configuration for Native Routing with Host-facing Ports as Access 
nv set interface lo ip address 10.10.10.1/32
nv set interface lo ip address 2010:10:10::1/128
nv set interface vlan100 type svi
nv set interface vlan100 vlan 100
nv set interface vlan100 base-interface br_default
nv set interface vlan100 ip address 2030:30:30::1/64
nv set interface vlan100 ip address 30.30.30.1/24
nv set bridge domain br_default vlan 100
nv set interface pf0hpf_if,pf1hpf_if bridge domain br_default access 100
nv set vrf default router bgp router-id 10.10.10.1
nv set vrf default router bgp autonomous-system 65501
nv set vrf default router bgp path-selection multipath aspath-ignore on
nv set vrf default router bgp address-family ipv4-unicast enable on
nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default router bgp address-family ipv6-unicast enable on
nv set vrf default router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf default router bgp neighbor p0_if remote-as external
nv set vrf default router bgp neighbor p0_if type unnumbered
nv set vrf default router bgp neighbor p0_if address-family ipv4-unicast enable on
nv set vrf default router bgp neighbor p0_if address-family ipv6-unicast enable on
nv set vrf default router bgp neighbor p1_if remote-as external
nv set vrf default router bgp neighbor p1_if type unnumbered
nv set vrf default router bgp neighbor p1_if address-family ipv4-unicast enable on
nv set vrf default router bgp neighbor p1_if address-family ipv6-unicast enable on
Sample NVUE Configuration for Native Routing with Host-facing Ports as Trunk 
nv set interface lo ip address 10.10.10.1/32
nv set interface lo ip address 2010:10:10::1/128
nv set interface vlan100 type svi
nv set interface vlan100 vlan 100
nv set interface vlan100 base-interface br_default
nv set interface vlan100 ip address 2030:30:30::1/64
nv set interface vlan100 ip address 30.30.30.1/24
nv set bridge domain br_default vlan 100
nv set interface pf0hpf_if,pf1hpf_if bridge domain br_default
nv set vrf default router bgp router-id 10.10.10.1
nv set vrf default router bgp autonomous-system 65501
nv set vrf default router bgp path-selection multipath aspath-ignore on
nv set vrf default router bgp address-family ipv4-unicast enable on
nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default router bgp address-family ipv6-unicast enable on
nv set vrf default router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf default router bgp neighbor p0_if remote-as external
nv set vrf default router bgp neighbor p0_if type unnumbered
nv set vrf default router bgp neighbor p0_if address-family ipv4-unicast enable on
nv set vrf default router bgp neighbor p0_if address-family ipv6-unicast enable on
nv set vrf default router bgp neighbor p1_if remote-as external
nv set vrf default router bgp neighbor p1_if type unnumbered
nv set vrf default router bgp neighbor p1_if address-family ipv4-unicast enable on
nv set vrf default router bgp neighbor p1_if address-family ipv6-unicast enable on
Sample Flat Files Configuration for Native Routing with Host-facing Ports as Access

Example /etc/network/interfaces configuration:

auto lo
iface lo inet loopback
    address 10.10.10.1/32
    address 2010:10:10::1/128

auto p0_if
iface p0_if

auto p1_if
iface p1_if

auto pf0hpf_if
iface pf0hpf_if
	bridge-access 100

auto pf1hpf_if
iface pf1hpf_if
 	bridge-access 100

auto vlan100
iface vlan100
    address 2030:30:30::1/64
    address 30.30.30.1/24
    vlan-raw-device br_default
    vlan-id 100

auto br_default
iface br_default
    bridge-ports pf0hpf_if pf1hpf_if
    bridge-vlan-aware yes
    bridge-vids 100
    bridge-pvid 1
Sample Flat Files Configuration for Native Routing with Host-facing Ports as Trunk

Example /etc/network/interfaces configuration:

auto lo
iface lo inet loopback
    address 10.10.10.1/32
    address 2010:10:10::1/128

auto p0_if
iface p0_if

auto p1_if
iface p1_if

auto pf0hpf_if
iface pf0hpf_if
	<

Last updated: