DOCA SDK Documentation

DOCA-Host Installation and Upgrade


The following options are available for installing or upgrading DOCA on host:

Method

Option Name

Description

Installation

DOCA local repo package

This option installs DOCA components

Upgrade

DOCA online repo

This option upgrades DOCA components using online repo from the Internet

Find the relevant DOCA installation files for your host in the NVIDIA DOCA Downloads page.

This page provides basic instructions for setting up DOCA on a standard system. For device-specific setups, refer to the respective device documentation, as additional steps may be required.

Uninstalling Software from Host

If an older DOCA (or MLNX_OFED) software version is installed on your host, make sure to uninstall it before proceeding with the installation of the new version:

Deb-based


$ for f in $( dpkg --list | grep -E 'doca|flexio|dpa-gdbserver|dpa-stats|dpaeumgmt' | awk '{print $2}' ); do echo $f ; apt remove --purge $f -y ; done
$ /usr/sbin/ofed_uninstall.sh --force
$ sudo apt-get autoremove


RPM-based


host# for f in $(rpm -qa | grep -i doca ) ; do yum -y remove $f; done
host# /usr/sbin/ofed_uninstall.sh --force
host# yum autoremove
host# yum makecache



Package and Kernel Module Signing

This section highlights the signing methods used to ensure the authenticity and integrity of the packages and kernel modules.

Package Signing

RPM packages are signed with a GPG key, enabling users to confirm that the packages come from a trusted NVIDIA source and have not been altered.

During installation, package managers automatically fetch and verify the signature while updating their metadata cache. 

Kernel Module Signing

Kernel drivers are signed using NVIDIA's x.509 public key, allowing users to verify that the drivers originate from a trusted NVIDIA source and have not been tampered with.

To support loading DOCA-HOST drivers on a secure-boot-enabled OS, the NVIDIA x.509 public key should be added to the key database using the following steps:

  1. Obtain NVIDIA's public x.509 key:

    # wget http://www.mellanox.com/downloads/ofed/mlnx_signing_key_pub.der 
    


    Builds for SLES15 SP4 and SP5 are signed with a newer signing key. The corresponding public key can be downloaded from this link.


  2. Import the public key to the MOK list using the mokutil utility:

    # mokutil --import mlnx_signing_key_pub.der 
    


    Users will be asked to enter and confirm a password for this MOK enrollment request.


  3. Reboot the system.

Installing Software on Host

  1. Install DOCA local repo package for host:

    • For deb-based distributions:


  2. Load the drivers:

    host# sudo /etc/init.d/openibd restart
    
  3. Initialize MST:

    host# sudo mst restart
    


DOCA Extra Package and doca-kernel-support

If the kernel version on your host is not supported (not shown under section "Supported Host OS per DOCA-Host Installation Profile"), users may opt to either:

  • Switch to a compatible kernel; or

  • Install doca-extra package and run doca-kernel-support

doca-kernel-support is a script intended to rebuild kernel modules included with DOCA-host for cases where they are not provided. It builds a single package (i.e., rpm or deb) that contains a repository of packages (i.e., dnf or apt).

DOCA-host provides binary builds of kernel modules for some specific kernels. This script rebuilds kernel modules included with DOCA-host for a custom kernel version and creates an RPM or DEB package that holds all of those rebuilt modules for easy install.

doca-kernel-support does not support customized or unofficial kernels.

To run doca-kernel-support:

  1. Run:

    host# apt/yum install -y doca-extra
    
  2. Execute the doca-kernel-support script which rebuilds and installs the DOCA-host kernel modules with the running kernel:

    host# /opt/mellanox/doca/tools/doca-kernel-support
    

    The output may end with rpm or deb with the following general format:

    doca-kernel-support: Built single package: <doca-kernel-repo>
    doca-kernel-support: Done
    

    Output example: 

    doca-kernel-support: Built single package: /tmp/DOCA.EuUfkWfV7Z/doca-kernel-repo-2.9.0-1.kver.5.14.0.356.el9.x86.64.x86_64.rpm
    doca-kernel-support: Done
    

    doca-kernel-support does not support customized or unofficial kernels.

  3. Install the generated meta-package on the desired DOCA profile:

    The examples provided install DOCA kernel repos (RPM/DEB) with doca-ofed profile, but other profiles may be installed if they are supported.

    • For RPM-based distributions:

      host# rpm -i <doca-kernel-repo>
      host# dnf makecache
      host# dnf install <doca-userspace-metapackage>   
      host# dnf install --disablerepo=doca doca-kernel-KERNEL_VERSION     # For all the meta-package, not just this single package					
      

      Example: 

      host# rpm -i /tmp/DOCA.EuUfkWfV7Z/doca-kernel-repo-2.9.0-1.kver.5.14.0.356.el9.x86.64.x86_64.rpm
      host# dnf makecache
      host# dnf install doca-ofed-userspace
      host# dnf install --disablerepo=doca doca-kernel-5.14.0.356.el9.x86.64.x86_64.noarch
      


    • For DEB-based distributions: 

      host# dpkg -i <doca-kernel-repo>
      host# apt update
      host# apt install <doca-profile>
      

      Example: 

      host# dpkg -i /tmp/DOCA.J8JYxEmffD/doca-kernel-repo-2.9.0-6.4.0.mlnx_2.9.0_amd64.deb
      host# apt update
      host# apt install doca-ofed
      
  4. (Optional) Retrieve installed packages and their versions as part of DOCA Host installation: 

    host# /opt/mellanox/doca/tools/doca-info
    
    Versions:
    - DOCA Base MLNX_OFED_LINUX-24.07-0.5.5.0
    - MFT 4.29.0-127
     
    UEFI\ATF versions:
    - mst_device: mt41692_pciconf0
         UEFI Version: 4.7.0-42-g13081ae
         ATF Version: 4.7.0-25-g5569834
    
    Firmware (Current):
    - BlueField-3 32.41.1000
     
    DOCA:
    - doca-all 2.8.0-0.0.4
    - doca-apsh-config 2.8.0079-1
    - doca-bench 2.8.0079-1
    …
     
    DOCA Dependencies:
    …
    - flexio 24.07.2300
    - mlnx-dpdk 22.11.0-2407.0.10
     
     
    OFED:
    …
    - rdma-core 2407mlnx52-1.2407055
    …
    - ucx 1.17.0-1.2407055
    …
    

    If BlueField has a BF-Bundle version older than 2.7.0 installed on it, the UEFI/ATF versions would appear as N/A. If your version is 2.7.0 or higher and still see N/A, then perform driver restart on the host:

    /etc/init.d/openibd restart
    

Upgrading Software on Host

To update existing DOCA host packages, follow these steps:

  1. Install the newer version of the DOCA host repository as detailed in section "DOCA-Host Installation and Upgrade | id (2.10.0)DOCA HostInstallationandUpgrade InstallingSoftwareonHost".

  2. Upgrade the DOCA packages:

    • To upgrade from DOCA version 2.5.x, all DOCA and OFED related packages should be removed. For detailed instructions on how to uninstall these packages, please refer to the uninstall section "DOCA-Host Installation and Upgrade | id (2.10.0)DOCA HostInstallationandUpgrade UninstallingSoftwarefromHost".

    • To upgrade from DOCA 2.6.0 and later, use the following command:

      • For DEB-based distributions:

        host# apt install doca-all
        


      • For RPM-based distributions:

        host# yum upgrade doca-all

    • Before upgrading mlnx-fw-updater, make sure to restart MST first:

      host# mst restart

Installation on Community Operating Systems

In the community model, there is shared responsibility between NVIDIA and customers choosing to use community OSs in their environment:

  • NVIDIA owns basic validation for the OSs, so that customers know they can expect DOCA to work.

  • Customers are responsible for building their own packages and binaries (based on source code and build instructions detailed below), and can also choose to deploy parts of DOCA instead of the whole package

NVIDIA offers support to customers and partners with active Support contracts. However, in some cases, NVIDIA may require customers to collaborate with the open-source community to address issues determined to be caused by community changes that disrupt DOCA functionality.

To install doca-host-community on a host, follow these steps:

  1. Install the doca-host-community repository that includes sources and tools to generate the DOCA community repo:

    • For RPM-based packages:

      host# rpm -uvh doca-host-repo-community-{doca_version}.noarch.rpm
      

      Example:

      host# rpm -uvh doca-host-repo-community-2.9.0-0.2.9.24.10.0.5.2.0.noarch.rpm
      


    • For DEB-based packages:

      host# dpkg -i doca-host_{doca_version}-community_all.deb
      

      Example:

      host# dpkg -i doca-host_2.9.0-100000-24.10-community_all.deb
      


  2. Run the build script to generate the doca-host-community repository:

    host# /opt/mellanox/doca/tools/doca-community-build
    

    The script output should include a line similar to the following:

    • For RPM-based packages:

      host# doca-community-build: Built single package: /tmp/DOCA.Lz1pntWcGM/doca-community-repo-24.10.0.5.2.0-1.kver.4.18.0.477.10.1.el8.8.x86.64.x86_64.rpm
      


    • For DEB-based packages:

      host# doca-community-build: Built single package: /tmp/DOCA.gcVyNokLfV/doca-community-repo-24.10-0.5.2.0-6.8.0.31.generic_24.10.0.5.2.0_amd64.deb
      


      The binary created by the script can be copied to any similar machine with the same kernel. It contains all of the doca-ofed profile built packages, with extra meta-package that depends on those packages, installing the meta-package will install all of the built packages.


  3. The resulting DOCA community repo should be ready for installation on this host or distributed to any other similar machine with the same distribution and kernel:

    • For RPM-based (non-SLES) packages:

      host# yum install /tmp/DOCA.m6rIcEJNKl/doca-community-repo-24.10.0.4.6.0-1.kver.5.14.0.427.13.1.el9.4.x86.64.x86_64.rpm
      host# yum makecache
      


    • For SLES RPM-based packages:

      host# zypper install /tmp/DOCA.m6rIcEJNKl/doca-community-repo-24.10.0.4.6.0-1.kver.5.14.0.427.13.1.el9.4.x86.64.x86_64.rpm
      host# zypper refresh
      


    • For DEB-based packages:

      host# apt install /tmp/DOCA.gcVyNokLfV/doca-community-repo-24.10-0.5.2.0-6.8.0.31.generic_24.10.0.5.2.0_amd64.deb
      host# apt update
      


  4. Install doca-ofed-community meta-package:

    host# yum/apt install doca-ofed-community
    


Installing Proprietary Packages

The installation procedure does not install proprietary packages. Those packages are installed upon request.

List of close-source proprietary packages:

  • Clusterkit

  • DPCP

  • hcoll

  • sharp

  • ibutils2

  • opensm

Currently, the only way to install these packages is by using an already-built RPM or DEB file from a similar primary OS.

The following table maps community OSs which are most similar to primary OSs:

Community OS

Most Similar Primary OS

Alma 8.5

RHEL 8.5

Anolis OS 8.4

RHEL 8.5

CentOS Stream 8

RHEL 8

CentOS Stream 9

RHEL 9

EulerOS-V2.0.SP10

EulerOS-V2.0.SP11

Fedora 35

RHEL 8.5

OpenEuler-20.03.SP1

OpenEuler20 SP3

OpenSUSE 15.3

SLES15 SP3

Photon OS 3.0

RHEL 7.9

UOS-V20-1040d

Debian 10.8

Download and Install MFT and Firmware

Storage Installation

  1. Before installation, search for the required package using your system's package manager:

    # apt/zypper/dnf/yum search mlnx-nvme
    # apt/zypper/dnf/yum search mlnx-nfsrdma

  2. Once the correct package name is identified, install it using:

    # apt/zypper/dnf/yum install <full-package-name>

Installation examples:

  • For SLES:

    # zypper search mlnx-nvme
    # zypper install mlnx-nvme-kmp-default

  • For Debian-based systems:

    # apt search mlnx-nvme
    # apt search mlnx-nfsrdma
    # apt install mlnx-nvme-dkms
    # apt install mlnx-nfsrdma-dkms

Last updated: