Networking NVIDIA UFM Enterprise Appliance Software User Manual
Networking NVIDIA UFM Enterprise Appliance Software User Manual
Breadcrumbs

UFM Enterprise Appliance Upgrade

Upgrading UFM Enterprise appliance is supported only from up to two previous GA versions (GA-1 or GA-2). For example, if you wish to upgrade to UFM Enterprise v1.17.1, it is possible to do so only from UFM Enterprise v1.16.1 or v1.15.1.

Upgrading UFM Enterprise appliance from an LTS version to a GA version, and vice versa, is not supported. Users can upgrade from GA-1 or GA-2 to the selected GA version, or — in the case of a UFM LTS version — from a specific LTS update version to a newer update on the same LTS (for example, upgrading from LTS-2024-U1 to LTS-2024-U3).

This is the recommended upgrade procedure, which involves upgrading all UFM Enterprise appliance software components and operating system. For additional upgrade procedures of specific software components, please refer to Appendix - Software Components Upgrade. 

As of UFM Enterprise Appliance version 1.5.0, upgrading the appliance on HA supports an in-service upgrade, meaning UFM can continue running during the steps of the upgrade, and there is no need to stop UFM before the upgrade.

As of version UFM Enterprise Appliance v1.10.0, some plugin images are available for deployment as part of the OMU (OS Manufacturer Upgrade) image. Information regarding the relevant plugins and their versions is available in the "show version" CLI command and in the /etc/ufm-release file.
The unique image name is "/var/opt/ufm/images/ufm-plugin-<plugin name>_from_release.tgz" to allow overriding previous plugin versions.
Note: Plugin versions are not upgraded automatically. The OMU upgrade only makes the new plugin versions available - upgrading them is a separate, manual step. For more information, refer to the UFM plugin upgrade command: UFM Commands | ufm plugin.

The unique image name is "/var/opt/ufm/images/ufm-plugin-<plugin name>_from_release-docker.img.gz" to allow overriding previous plugin versions.

In-Service Upgrade via SHELL

Upgrade docs omit mandatory manual plugin upgrade step and misleading plugin version verification

The upgrade is performed on both Master and Standby nodes.

To upgrade the UFM Enterprise Appliance software:

  1. On the standby server, extract the OMU image to the /tmp folder:

    tar -xzf ufm-appliance-<version>-omu.ubuntu24.tar -C /tmp

  2. On the standby server, access the installation folder and upgrade script:  

    standby# cd /tmp/ufm-appliance-<version>-omu.ubuntu24
:note:

If the iDRAC password is not available, comment out the BIOS upgrade command in ufm-os-upgrade.sh, (instead steps 3 and 6). Then, run the upgrade as listed in step 4 and onwards.

Bash
sed -i '/\/usr\/bin\/ufm_bios_upgrade\.sh >> \${logfile} 2>&1/s/^/# /' dev_internals/ufm-os-upgrade.sh

Note that in this scenario the BIOS upgrade is skipped.

  1. Disable the iDRAC System Lockdown Mode for successful BIOS upgrade, run:

    curl -sk -u "root:<idrac password>" -X PATCH -H "Content-Type: application/json" -d '{"Attributes":{"Lockdown.1.SystemLockdown":"Disabled"}}' "https://169.254.1.1/redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/DellAttributes/iDRAC.Embedded.1" | python3 -c "import sys, json; print(json.load(sys.stdin)['@Message.ExtendedInfo'][0]['Message'])"
:note:

Verify the iDRAC System Lockdown Mode is applied, run:

curl -sk -u "root:<idrac password>" "https://169.254.1.1/redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/DellAttributes/iDRAC.Embedded.1" | python3 -c "import sys, json; print(json.load(sys.stdin).get('Attributes', {}).get('Lockdown.1.SystemLockdown'))"


  1. Run the UFM upgrade script on the standby server:

    ./ufm-os-upgrade.sh --yes --reboot

  2. After the reboot procedure is complete, a systemd service (ufm-os-firstboot.service) runs the remainder of the upgrade procedure. Once completed, a message is prompted to all open terminals including the status:
    "UFM-OS-FIRSTBOOT-FAILURE" - if installation is failed.
    "UFM-OS-FIRSTBOOT-SUCCESS" - if installation succeeded.
    Example:

    root# ufm-ai03:~#
root# ufm-ai03:~#
Broadcast message from root@ufm-ai03 (somewhere) (Fri Dec 30 18:47:32 2022):

UFM-OS-FIRSTBOOT-SUCCESS, installation succeeded additional info is available in /var/log/ufm-os-firstboot.log

    To manually check the status, run systemctl status ufm-os-firstboot.service. If it is already finished, an error message is prompted stating that there is no such service. In that case, the log /var/log/ufm-os-firstboot.log can be checked instead.

    systemctl status ufm-os-firstboot.service

    Example: 

    root# ufm-ai03:~# systemctl status ufm-os-firstboot
Unit ufm-os-firstboot.service could not be found.
root# ufm-ai03:~#

Do NOT proceed to the next step before ensuring that the systemctl status ufm-os-firstboot.service service has been completed.

  1. Once completed, re-enable the iDRAC System Lockdown Mode:

    curl -sk -u "root:<idrac password>" -X PATCH -H "Content-Type: application/json" -d '{"Attributes":{"Lockdown.1.SystemLockdown":"Enabled"}}' "https://169.254.1.1/redfish/v1/Managers/iDRAC.Embedded.1/Oem/Dell/DellAttributes/iDRAC.Embedded.1" | python3 -c "import sys, json; print(json.load(sys.stdin)['@Message.ExtendedInfo'][0]['Message'])"

  2. After the completion of the upgrade script, the UFM code is upgraded, while the UFM data remains unchanged. The automatic upgrade of UFM data will take place during the next UFM startup. To initiate this process, execute a failover from the Master node (or perform a takeover from the Standby node).

    master# ufm_ha_cluster failover

The upgrade script logs the data to /var/log/ufm_os_upgrade_<UFM-OS version>.log and outputs simultaneously it to the screen. In case of an issue, UFM data can be restored to factory default. For more information, refer to Appendix - UFM Factory Reset.

  1. Once UFM is operational on the upgraded node (formerly the standby node), proceed to replicate steps 1 to 3 on the non-upgraded node (previously the master node).

In-Service Upgrade via CLI

Alternatively, in-service upgrade can be performed via the CLI. The upgrade is performed on both Master and Standby nodes.

Follow the below instructions:

  1. On the Standby node, fetch the new image from a remote server. Run: 

    ufmapl (config) # image fetch <download URL>

  2. On the Standby node, install the new image. Run:

    ufmapl (config) # image install <image name>

  3. Reload the Standby UFM Enterprise Appliance. Run:

    ufmapl (config) # reload

  4. After the completion of the upgrade on the Standby node, the UFM code is upgraded, while the UFM data remains unchanged. The automatic upgrade of UFM data will take place during the next UFM startup. To initiate this process, execute a failover from the Master node. Once the Standby node is up and running, perform a failover on the Master node. Run:

    ufmapl (config) # ufm ha failover

  5. Once UFM is operational on the upgraded node (formerly the standby node), proceed to replicate steps 1 to 3 on the non-upgraded node (previously the Master node).



Last updated: