DOCA SDK Documentation

BF-Bundle Installation and Upgrade


Important! Make sure to update DOCA on the host side before installing the BF-bundle on the BlueField.


Users have three options for installing or upgrading DOCA on BlueField DPU or SuperNIC:

Method

Option Name

Description

Installation

BF-bundle DOCA image

This option overwrites the entire boot partition with an Ubuntu 22.04 installation and updates BlueField and NIC firmware

Upgrade

Standard Linux Tools

This option upgrades DOCA components without overwriting the boot partition. Use this option to preserve configurations or files on BlueField itself

Installation

ISO DOCA image

This option overwrites the entire boot partition with an Ubuntu 22.04 installation


Find the DOCA installation files for BlueField from the NVIDIA DOCA Downloads page.

 Installing DOCA-Host

Make sure to update DOCA on the host side before installing the BF-bundle on the BlueField.

Installing Software on BlueField Using BF-Bundle

The ATF will not boot 150W BlueField-3 platforms if the ATX +12V power supply is not connected. This requirement ensures the proper operation of the BlueField device. For detailed instructions on connecting the external power supply connector, refer to the NVIDIA BlueField-3 Networking Platform User Guide.

Users can install DOCA on BlueField DPU or SuperNIC by upgrading the full DOCA image on BlueField. 

This overwrites the entire boot partition with an Ubuntu 22.04 installation and updates BlueField and NIC firmware.

This installation sets up the OVS bridge.

Option 1 – No Pre-defined Password

To change the default Ubuntu password during the BFB bundle installation, proceed to BF-Bundle Installation and Upgrade | id (2.10.0)BF BundleInstallationandUpgrade Option2.

To install a BFB, use the following command:

Bash
host# sudo bfb-install --rshim rshim<N> --bfb <image_path.bfb>

Where:

  • <N> – the appropriate RShim device identifier. If you have only one BlueField device, use rshim0.

  • <image_path.bfb> – Replace this with the path to the BFB image file.

To list the RShim devices present on your system, run the following command:

Bash
host# ls -la /dev/ | grep rshim

By default bfb-install will clear the RShim log in /dev/rshim<N>/misc and save it as tmp/bfb-install-rshim[N].log instead. To preserve the RShim log in /dev/rshim<N>/misc, provide the --keep-log argument to the bfb-install command line.

Option 2 – Set Pre-defined Password

Ubuntu users can set a unique password for the ubuntu user in the bf.cfg configuration file, which will be applied automatically at the end of the BlueField BFB bundle installation. 

To do this, follow these steps:

  1. Generate the hashed password: 

    host# openssl passwd -1
    Password:
    Verifying - Password:
    $1$3B0RIrfX$TlHry93NFUJzg3Nya00rE1
    


  2. Add the password hash to the bf.cfg configuration file: 

    host# echo ubuntu_PASSWORD='$1$3B0RIrfX$TlHry93NFUJzg3Nya00rE1' > bf.cfg
    


  3. Use the --config flag to specify the configuration file when running the installation command: 

    host# sudo bfb-install --rshim rshim<N> --bfb <image_path.bfb> --config bf.cfg
    

    If --config is not used, then upon first login to the BlueField device, users will be prompted to update the default ubuntu password.


    Optionally, to upgrade the BlueField integrated BMC firmware using BFB bundle, please provide the current BMC root credentials in a bf.cfg file, as shown in the following:

    BMC_PASSWORD="<root password>"
    BMC_USER="root"
    BMC_REBOOT="yes"
    

    Unless previously changed, the default BMC root password is 0penBmc.

    The following is an example output of Ubuntu-22.04 BFB bundle installation (version may vary in the future):

    host# sudo bfb-install --rshim rshim0 --bfb bf-bundle-2.7.0_24.04_ubuntu-22.04_prod.bfb --config bf.cfg
    Pushing bfb 1.41GiB 0:02:02 [11.7MiB/s] [           <=>                                                                                                                                ]
    Collecting BlueField booting status. Press Ctrl+C to stop
     INFO[PSC]: PSC BL1 START
     INFO[BL2]: start
     INFO[BL2]: boot mode (rshim)
     INFO[BL2]: VDDQ: 1120 mV
     INFO[BL2]: DDR POST passed
     INFO[BL2]: UEFI loaded
     INFO[BL31]: start
     INFO[BL31]: lifecycle GA Secured
     INFO[BL31]: VDD: 850 mV
     INFO[BL31]: runtime
     INFO[BL31]: MB ping success
     INFO[UEFI]: eMMC init
     INFO[UEFI]: eMMC probed
     INFO[UEFI]: UPVS valid
     INFO[UEFI]: PMI: updates started
     INFO[UEFI]: PMI: total updates: 1
     INFO[UEFI]: PMI: updates completed, status 0
     INFO[UEFI]: PCIe enum start
     INFO[UEFI]: PCIe enum end
     INFO[UEFI]: UEFI Secure Boot 
     INFO[UEFI]: PK configured
     INFO[UEFI]: Redfish enabled
     INFO[UEFI]: exit Boot Service
     INFO[MISC]: Found bf.cfg
     INFO[MISC]: Ubuntu installation started
     INFO[MISC]: Installing OS image
     INFO[MISC]: Changing the default password for user ubuntu
     INFO[MISC]: Ubuntu installation completed
     INFO[MISC]: Updating NIC firmware...
     INFO[MISC]: NIC firmware update done
     INFO[MISC]: Installation finished 
    
    


  4. To verify that the BlueField device has completed booting, wait 90 seconds after the installation process finishes and run the following command:

    host# sudo cat /dev/rshim<N>/misc
    ...
     INFO[MISC]: Linux up
     INFO[MISC]: DPU is ready  
    


  5. Retrieve installed packages and their versions as part of BF-Bundle installation:

    1. Log into BlueField.

    2. Run the following: 

      bf# sudo bf-info
      

      Example output: 

      Versions:
      - ATF: v2.2(release):4.9.0-16-g221717c68
      - UEFI: 4.9.0-37-gcbeaab0650
      - BSP: 4.9.0.13322
      - NIC Firmware: 32.43.0356
      - DOCA Base (OFED): 24.10-0.5.1.0
      ...
      Storage:
      - mlnx-libsnap 1.6.0-1
      - spdk 23.01.5-24
      - virtio-net-controller 24.10.15-1
      DOCA:
      - doca-apsh-config 2.9.0064-1
      - libdoca-sdk-urom-dev 2.9.0064-1
      ...
      FlexIO:
      - flexio-samples 24.10.2447
      - flexio-sdk 24.10.2447
      ...
      SoC Platform:
      - mlxbf-gige-modules 1.0-0.kver.6.1.0-11-arm64
      - sdhci-of-dwcmshc-modules 1.0-0.kver.6.1.0-11-arm64
      ...
      OFED:
      rdma-core 2410mlnx54-1.2410051
      ucx 1.18.0-1.2410051
      ...
      


  6. To configure the tmfifo_net0 interface over IPv4 for SSH access to the BlueField Arm OS:

    host# ifconfig tmfifo_net0 192.168.100.1/24

    SSH into the BlueField Arm OS with 192.168.100.2 (preconfigured default): 

    host# ssh ubuntu@192.168.100.2

Upgrading BlueField Using Standard Linux Tools

This procedure allows for upgrading DOCA components on BlueField networking platforms (DPUs or SuperNICs) using standard Linux tools (e.g., apt update and yum update). It leverages native package manager repositories to update components without requiring a full reinstallation.

Important!

While this approach enables updating specific DOCA components, the combinations created by selective updates are not validated by NVIDIA. NVIDIA only validates the full installation of the BF-Bundle package.

This process has the following benefits:

  • Only updates components that include modifications

  • Includes upgrade of:

    • DOCA drivers and libraries 

    • DOCA reference applications

    • BSP (UEFI/ATF) upgrade while maintaining the configuration

    • NIC firmware upgrade while maintaining the configuration

    • BMC components upgrade

  • Does not:

    • Impact user binaries

    • Upgrade non-Ubuntu OS kernels

  • After completion of BlueField upgrade:

    • If NIC firmware was not updated, perform BlueField Arm reset (software reset/reboot BlueField)

    • If NIC firmware was updated, perform firmware reset (mlxfwreset) or perform a graceful shutdown and power cycle

OS

Action

Instructions

Ubuntu/
Debian

Remove mlxbf-bootimages package


<bf> $ apt remove --purge mlxbf-bootimages* -y


Install the GPG key


<bf> $ apt update
<bf> $ apt install gnupg2


Export the desired distribution

Export DOCA_REPO with the relevant URL:

<bf> $ export DOCA_REPO="<URL>"

Add GPG key to APT trusted keyring


<bf> $ curl $DOCA_REPO/GPG-KEY-Mellanox.pub | gpg --dearmor > /etc/apt/trusted.gpg.d/GPG-KEY-Mellanox.pub


Add DOCA online repository


<bf> $ echo "deb [signed-by=/etc/apt/trusted.gpg.d/GPG-KEY-Mellanox.pub] $DOCA_REPO ./" > /etc/apt/sources.list.d/doca.list


Update index


<bf> $ apt update


Upgrade UEFI/ATF firmware

Install the ATF/UEFI package and mlxbf-scripts:

<bf> $ apt install mlxbf-bootimages-signed mlxbf-scripts

Initiate the UEFI/ATF firmware upgrade:

<bf> $ bfrec


Upgrade BlueField NIC firmware

Install the NIC firmware package:

<bf> $ apt install mlnx-fw-updater-signed

Initiate NIC firmware upgrade:

<bf> $ sudo /opt/mellanox/mlnx-fw-updater/mlnx_fw_updater.pl --force-fw-update


Upgrade BMC components

Insert the BMC password to config file /etc/bf-upgrade.conf:

The format of bf-upgrade.conf should be identical to bf.cfg. Refer to "Customizing BlueField Software Deployment Using bf.cfg" for information.


<bf> $ cat > /etc/bf-upgrade.conf << EOF
BMC_PASSWORD="<password>"
EOF

To apply the new firmware immediately after the BMC versions are upgraded, add the following parameters to the bf-upgrade.conf file:

<bf> $ cat >> /etc/bf-upgrade.conf << EOF
BMC_REBOOT="yes"
CEC_REBOOT="yes"
EOF


BMC_REBOOT="yes" reboots the BMC after the version update and resets the console.

CEC_REBOOT="yes" only works if the current CEC version is 00.02.0180.0000 and above.


To apply the new BMC firmware at a later time, users must reset the BMC and CEC components manually. Refer to "Resetting CEC and BMC Subsystems Using CEC Self-reset Command" for reference.

Install the packages of BMC components:

<bf> $ apt -y install bf3-bmc-nic-fw-* bf3-bmc-fw-signed bf3-cec-fw-signed bf3-bmc-gi-signed
<bf> $ apt -y install bf-release*

Initiate the upgrade of BMC components:

<bf> $ dpu-bmc-upgrade


The upgrade process should take up to 20 minutes.


Remove old metapackages


<bf> $ apt-get remove doca* mlnx-ofed* kernel-mft* -y


Install new metapackages


<bf> $ apt-get install doca-runtime doca-devel -y


Upgrade system


<bf> $ apt upgrade


Apply the new changes, NIC firmware, and UEFI/ATF 

For the upgrade to take effect, perform BlueField system reboot.

This step triggers immediate reboot of the BlueField Arm cores.


CentOS/RHEL/
Anolis/Rocky

Remove mlxbf-bootimages package


<bf> $ yum -y remove mlxbf-bootimages* --noautoremove
<bf> $ yum makecache


Export the desired distribution

Export DOCA_REPO with the relevant URL:

<bf> $ export DOCA_REPO="<URL>"

Add DOCA online repository


echo "[doca] 
name=DOCA Online Repo 
baseurl=$DOCA_REPO 
enabled=1 
gpgcheck=0 
priority=10 
cost=10" > /etc/yum.repos.d/doca.repo

A file is created under /etc/yum.repos.d/doca.repo.

Update index


<bf> $ yum makecache


Upgrade UEFI/ATF firmware

Install the ATF/UEFI package and mlxbf-bfscripts:

<bf> $ yum install -y mlxbf-bootimages-signed mlxbf-bfscripts

Initiate UEFI/ATF firmware upgrade:

<bf> $ bfrec


Upgrade BlueField NIC firmware

The following command updates the firmware package and flashes the firmware to the NIC:

<bf> $ yum install -y mlnx-fw-updater-signed


Upgrade BMC components 

Insert the BMC password to config file /etc/bf-upgrade.conf:

The format of bf-upgrade.conf should be identical to bf.cfg. Refer to "Customizing BlueField Software Deployment Using bf.cfg" for reference.


<bf> $ cat > /etc/bf-upgrade.conf << EOF
BMC_PASSWORD="<password>"
EOF

To apply the new firmware immediately after the BMC versions are upgraded, add the following parameters to the bf-upgrade.conf file:

<bf> $ cat >> /etc/bf-upgrade.conf << EOF
BMC_REBOOT="yes"
CEC_REBOOT="yes"
EOF


BMC_REBOOT="yes" reboots the BMC after the version update and resets the console.

CEC_REBOOT="yes" only works if the current CEC version is 00.02.0180.0000 and above.


To apply the new BMC firmware at a later time, users must reset the BMC and CEC components manually. Refer to "Resetting CEC and BMC Subsystems Using CEC Self-reset Command" for information.

Install the packages of BMC components:

<bf> $ yum -y install bf3-bmc-nic-fw-* bf3-bmc-fw-signed bf3-cec-fw-signed bf3-bmc-gi-signed
<bf> $ yum -y install bf-release*

Initiate the upgrade of BMC components:

<bf> $ dpu-bmc-upgrade


The upgrade process should take up to 20 minutes.


Remove old metapackages


<bf> $ yum remove doca*  mlnx-ofed* kernel-mft* -y


If you are upgrading from DOCA 2.7.0 or below, make sure to also remove strongSwan and Libreswan packages:

<bf> $ yum remove strongswan-bf strongswan-swanctl
<bf> $ yum remove strongswan-bf strongswan-swanctl libreswan



Install new metapackages


<bf> $ yum -y install doca-runtime doca-devel


For Anolis OS only, lock the kernel

Pin the kernel package versions:

<bf> dnf install python3-dnf-plugin-versionlock
<bf> dnf versionlock kernel*


Upgrade system


<bf> $ yum upgrade --nobest


For Anolis OS only, update grub file configuration

Update grub file parameter to enable the OS to boot after BlueField system reboot: 

<bf> $ sed -i 's/GRUB_ENABLE_BLSCFG=true/GRUB_ENABLE_BLSCFG=false/' /etc/default/grub; mv /boot/efi/EFI/anolis/grub.cfg.rpmsave /boot/efi/EFI/anolis/grub.cfg


Apply the new changes, NIC firmware, and UEFI/ATF

For the upgrade to take effect, perform BlueField system reboot.

This step triggers immediate reboot of the BlueField Arm cores.


Building Custom BFB Installation Image

Users wishing to build their own customized BlueField OS image can use the BFB build environment. For more details, refer to the bfb-build project in this GitHub webpage

To boot a customized BlueField OS image on a UEFI secure-boot-enabled BlueField (the default secure boot setting), the following conditions must be met:

  • The OS image must be signed with a key already present in the UEFI DB (e.g., the Microsoft key)

  • If the above is not possible, UEFI secure boot must be disabled to allow booting of an unsigned or differently signed image

For detailed instructions on managing UEFI secure boot, refer to the "Secure Boot" page in the BlueField Platform SW Documentation.

Last updated: