InfiniBand Cluster Bring-up Procedure

UFM Configuration

Configure a Static IPoIB on the IB Port:

  1. From master, stop UFM.

    ufm_ha_cluster stop
    


  2. On both servers, set a static IP address for the main IB interface (using the proper tool, e.g. netplan for Ubuntu).

  3. From master, start UFM.

    ufm_ha_cluster start
    


  4. Verify the configured IP address appears for the main IB interface using 'ifconfig'.

Recommended QoS Configuration:

Edit the following parameters in the OpenSM configuration file:

vi /opt/ufm/files/conf/opensm/opensm.conf


# Limit the maximal operational VLs
max_op_vls 2
# Enable QoS setup:
qos TRUE
# QoS policy file to be used:
qos_policy_file (null)
# Suppress QoS MAD status errors:
suppress_sl2vl_mad_status_errors FALSE
# Override multicast SL provided in join/create request:
override_create_mcg_sl 0xff
# QoS default options
qos_max_vls 0
qos_high_limit -1
qos_vlarb_high (null)
qos_vlarb_low (null)
qos_sl2vl 0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1

After the change of the OpenSM configuration file stop and restart the  ufm_ha_cluster 

ufm_ha_cluster stop 
ufm_ha_cluster start


For additional optional configuration, see Optional Configurations.

To upgrade UFM software, see Upgrading UFM Software.

Last updated: