Important! Make sure to update DOCA on the host side before installing the BF-bundle on the BlueField.
Users have three options for installing or upgrading DOCA on BlueField DPU or SuperNIC:
|
Method |
Option Name |
Description |
|---|---|---|
|
Installation |
This option overwrites the entire boot partition with an Ubuntu 22.04 installation and updates BlueField and NIC firmware |
|
|
Upgrade |
This option upgrades DOCA components without overwriting the boot partition. Use this option to preserve configurations or files on BlueField itself |
|
|
Installation |
ISO DOCA image |
This option overwrites the entire boot partition with an Ubuntu 22.04 installation |
Find the DOCA installation files for BlueField from the NVIDIA DOCA Downloads page.
Installing DOCA-Host
Make sure to update DOCA on the host side before installing the BF-bundle on the BlueField.
Installing Software on BlueField Using BF-Bundle
The ATF will not boot 150W BlueField-3 platforms if the ATX +12V power supply is not connected. This requirement ensures the proper operation of the BlueField device. For detailed instructions on connecting the external power supply connector, refer to the NVIDIA BlueField-3 Networking Platform User Guide.
Users can install DOCA on BlueField DPU or SuperNIC by upgrading the full DOCA image on BlueField.
This overwrites the entire boot partition with an Ubuntu 22.04 installation and updates BlueField and NIC firmware.
This installation sets up the OVS bridge.
Option 1 – No Pre-defined Password
To change the default Ubuntu password during the BFB bundle installation, proceed to BF-Bundle Installation and Upgrade | id (2.10.0)BF BundleInstallationandUpgrade Option2.
To install a BFB, use the following command:
host# sudo bfb-install --rshim rshim<N> --bfb <image_path.bfb>
Where:
-
<N>– the appropriate RShim device identifier. If you have only one BlueField device, usershim0. -
<image_path.bfb>– Replace this with the path to the BFB image file.
To list the RShim devices present on your system, run the following command:
host# ls -la /dev/ | grep rshim
By default bfb-install will clear the RShim log in /dev/rshim<N>/misc and save it as tmp/bfb-install-rshim[N].log instead. To preserve the RShim log in /dev/rshim<N>/misc, provide the --keep-log argument to the bfb-install command line.
Option 2 – Set Pre-defined Password
Ubuntu users can set a unique password for the ubuntu user in the bf.cfg configuration file, which will be applied automatically at the end of the BlueField BFB bundle installation.
To do this, follow these steps:
-
Generate the hashed password:
host# openssl passwd -1 Password: Verifying - Password: $1$3B0RIrfX$TlHry93NFUJzg3Nya00rE1
-
Add the password hash to the
bf.cfgconfiguration file:host# echo ubuntu_PASSWORD='$1$3B0RIrfX$TlHry93NFUJzg3Nya00rE1' > bf.cfg
-
Use the
--configflag to specify the configuration file when running the installation command:host# sudo bfb-install --rshim rshim<N> --bfb <image_path.bfb> --config bf.cfgIf
--configis not used, then upon first login to the BlueField device, users will be prompted to update the defaultubuntupassword.
Optionally, to upgrade the BlueField integrated BMC firmware using BFB bundle, please provide the current BMC root credentials in a
bf.cfgfile, as shown in the following:BMC_PASSWORD="<root password>" BMC_USER="root" BMC_REBOOT="yes"Unless previously changed, the default BMC root password is
0penBmc.The following is an example output of Ubuntu-22.04 BFB bundle installation (version may vary in the future):
host# sudo bfb-install --rshim rshim0 --bfb bf-bundle-2.7.0_24.04_ubuntu-22.04_prod.bfb --config bf.cfg Pushing bfb 1.41GiB 0:02:02 [11.7MiB/s] [ <=> ] Collecting BlueField booting status. Press Ctrl+C to stop INFO[PSC]: PSC BL1 START INFO[BL2]: start INFO[BL2]: boot mode (rshim) INFO[BL2]: VDDQ: 1120 mV INFO[BL2]: DDR POST passed INFO[BL2]: UEFI loaded INFO[BL31]: start INFO[BL31]: lifecycle GA Secured INFO[BL31]: VDD: 850 mV INFO[BL31]: runtime INFO[BL31]: MB ping success INFO[UEFI]: eMMC init INFO[UEFI]: eMMC probed INFO[UEFI]: UPVS valid INFO[UEFI]: PMI: updates started INFO[UEFI]: PMI: total updates: 1 INFO[UEFI]: PMI: updates completed, status 0 INFO[UEFI]: PCIe enum start INFO[UEFI]: PCIe enum end INFO[UEFI]: UEFI Secure Boot INFO[UEFI]: PK configured INFO[UEFI]: Redfish enabled INFO[UEFI]: exit Boot Service INFO[MISC]: Found bf.cfg INFO[MISC]: Ubuntu installation started INFO[MISC]: Installing OS image INFO[MISC]: Changing the default password for user ubuntu INFO[MISC]: Ubuntu installation completed INFO[MISC]: Updating NIC firmware... INFO[MISC]: NIC firmware update done INFO[MISC]: Installation finished
-
To verify that the BlueField device has completed booting, wait 90 seconds after the installation process finishes and run the following command:
host# sudo cat /dev/rshim<N>/misc ... INFO[MISC]: Linux up INFO[MISC]: DPU is ready
-
Retrieve installed packages and their versions as part of BF-Bundle installation:
-
Log into BlueField.
-
Run the following:
bf# sudo bf-infoExample output:
Versions: - ATF: v2.2(release):4.9.0-16-g221717c68 - UEFI: 4.9.0-37-gcbeaab0650 - BSP: 4.9.0.13322 - NIC Firmware: 32.43.0356 - DOCA Base (OFED): 24.10-0.5.1.0 ... Storage: - mlnx-libsnap 1.6.0-1 - spdk 23.01.5-24 - virtio-net-controller 24.10.15-1 DOCA: - doca-apsh-config 2.9.0064-1 - libdoca-sdk-urom-dev 2.9.0064-1 ... FlexIO: - flexio-samples 24.10.2447 - flexio-sdk 24.10.2447 ... SoC Platform: - mlxbf-gige-modules 1.0-0.kver.6.1.0-11-arm64 - sdhci-of-dwcmshc-modules 1.0-0.kver.6.1.0-11-arm64 ... OFED: rdma-core 2410mlnx54-1.2410051 ucx 1.18.0-1.2410051 ...
-
-
To configure the
tmfifo_net0interface over IPv4 for SSH access to the BlueField Arm OS:host# ifconfig tmfifo_net0 192.168.100.1/24
Upgrading BlueField Using Standard Linux Tools
This procedure allows for upgrading DOCA components on BlueField networking platforms (DPUs or SuperNICs) using standard Linux tools (e.g., apt update and yum update). It leverages native package manager repositories to update components without requiring a full reinstallation.
While this approach enables updating specific DOCA components, the combinations created by selective updates are not validated by NVIDIA. NVIDIA only validates the full installation of the BF-Bundle package.
This process has the following benefits:
-
Only updates components that include modifications
-
Includes upgrade of:
-
DOCA drivers and libraries
-
DOCA reference applications
-
BSP (UEFI/ATF) upgrade while maintaining the configuration
-
NIC firmware upgrade while maintaining the configuration
-
BMC components upgrade
-
-
Does not:
-
Impact user binaries
-
Upgrade non-Ubuntu OS kernels
-
-
After completion of BlueField upgrade:
-
If NIC firmware was not updated, perform BlueField Arm reset (software reset/reboot BlueField)
-
If NIC firmware was updated, perform firmware reset (
mlxfwreset) or perform a graceful shutdown and power cycle
-
|
OS |
Action |
Instructions |
|---|---|---|
|
Ubuntu/
|
Remove |
|
|
Install the GPG key |
|
|
|
Export the desired distribution |
Export
|
|
|
Add GPG key to APT trusted keyring |
|
|
|
Add DOCA online repository |
|
|
|
Update index |
|
|
|
Upgrade UEFI/ATF firmware |
Install the ATF/UEFI package and
Initiate the UEFI/ATF firmware upgrade:
|
|
|
Upgrade BlueField NIC firmware |
Install the NIC firmware package:
Initiate NIC firmware upgrade:
|
|
|
Upgrade BMC components |
Insert the BMC password to config file The format of
To apply the new firmware immediately after the BMC versions are upgraded, add the following parameters to the
To apply the new BMC firmware at a later time, users must reset the BMC and CEC components manually. Refer to "Resetting CEC and BMC Subsystems Using CEC Self-reset Command" for reference. Install the packages of BMC components:
Initiate the upgrade of BMC components:
The upgrade process should take up to 20 minutes.
|
|
|
Remove old metapackages |
|
|
|
Install new metapackages |
|
|
|
Upgrade system |
|
|
|
Apply the new changes, NIC firmware, and UEFI/ATF |
For the upgrade to take effect, perform BlueField system reboot. This step triggers immediate reboot of the BlueField Arm cores.
|
|
|
CentOS/RHEL/
|
Remove |
|
|
Export the desired distribution |
Export
|
|
|
Add DOCA online repository |
A file is created under |
|
|
Update index |
|
|
|
Upgrade UEFI/ATF firmware |
Install the ATF/UEFI package and
Initiate UEFI/ATF firmware upgrade:
|
|
|
Upgrade BlueField NIC firmware |
The following command updates the firmware package and flashes the firmware to the NIC:
|
|
|
Upgrade BMC components |
Insert the BMC password to config file The format of
To apply the new firmware immediately after the BMC versions are upgraded, add the following parameters to the
To apply the new BMC firmware at a later time, users must reset the BMC and CEC components manually. Refer to "Resetting CEC and BMC Subsystems Using CEC Self-reset Command" for information. Install the packages of BMC components:
Initiate the upgrade of BMC components:
The upgrade process should take up to 20 minutes.
|
|
|
Remove old metapackages |
If you are upgrading from DOCA 2.7.0 or below, make sure to also remove strongSwan and Libreswan packages:
|
|
|
Install new metapackages |
|
|
|
For Anolis OS only, lock the kernel |
Pin the kernel package versions:
|
|
|
Upgrade system |
|
|
|
For Anolis OS only, update grub file configuration |
Update grub file parameter to enable the OS to boot after BlueField system reboot:
|
|
|
Apply the new changes, NIC firmware, and UEFI/ATF |
For the upgrade to take effect, perform BlueField system reboot. This step triggers immediate reboot of the BlueField Arm cores.
|
Building Custom BFB Installation Image
Users wishing to build their own customized BlueField OS image can use the BFB build environment. For more details, refer to the bfb-build project in this GitHub webpage.
To boot a customized BlueField OS image on a UEFI secure-boot-enabled BlueField (the default secure boot setting), the following conditions must be met:
-
The OS image must be signed with a key already present in the UEFI DB (e.g., the Microsoft key)
-
If the above is not possible, UEFI secure boot must be disabled to allow booting of an unsigned or differently signed image
For detailed instructions on managing UEFI secure boot, refer to the "Secure Boot" page in the BlueField Platform SW Documentation.
Last updated: