ConnectX-8 Firmware Release Notes

Known Issues

VF Network Function Limitations in SR-IOV Legacy Mode & in Switchdev Mode

Dual Port Device

Single Port Device

127 VF per PF (254 functions)

127

 VF+SF Network Function Limitations in Switchdev Mode

Dual Port Device

Single Port Device

  • 127 VF per PF (254 functions)

  • 512 PF+VF+SF per PF (1024 functions)

  • 127 VF (127 functions)

  • 512 PF+VF+SF per PF (512 functions)

ConnectX-8 has the same feature set and limitations as ConnectX-7 adapter card. For the list of ConnectX-7 Known Issues, please go to https://docs.nvidia.com/networking/software/adapter-firmware/index.html#connectx-7.

The below are limitations related to ConnectX-8 only.

Internal Ref.

Issue

4893639

Description: Direct Memory Access (DMA) protection is not supported in the ipxe.efi driver.

Workaround: N/A

Keywords: DMA

Detected in version:

40.48.1000

4835832

Description: In rare cases, certain module types may experience link-up failures.

Workaround: Configure the requested speed using the following commands:

mlxconfig -d <device> set PHY_RATE_MASK_OVERRIDE_P<port>=1
mlxfwreset -d <device> r -y --no_mst
mlxconfig -d <device> set PHY_RATE_MASK_P<port>=<PTYS.speed_mask> (speed mask is defined in PRM)
mlxfwreset -d <device> r -y --no_mst

Keywords: Cables

Detected in version:

40.48.1000

4594515 / 4622127 / NVbug 5460310

Description: Running perftest with ib_send_bw using a message size of 4KB and 1000 QPs does not reach the expected full line rate.

Workaround: N/A

Keywords: perftest

Detected in version:

40.48.1000

4604969

Description: Probe packets might be dropped at the transmission stage when multiple congestion control flows are active.

Workaround: N/A

Keywords: PCC, RTT, probe

Detected in version:

40.47.1026

4496642

Description: The timestamps (t2, t4) of the received RTT probes are taken from the free-running clock, even when ROCE_CC_RTT_TIMESTAMP_FORMAT is set to 0x02.
The format of all RTT probe timestamps can be found in HCA_CAP.rtt_timestamp_format.

Workaround: N/A

Keywords: RTT RTC timestamp

Detected in version:

40.47.1026

4705241

Description: When Quantum-2 is part of an XDR topology, serving as a leaf switch connected to NDR-based hosts, a bandwidth degradation of approximately 3–7 Gb/s is expected.

Workaround: N/A

Keywords: XDR, NDR, Quantum-2

Detected in version:

40.47.1026

4705948

Description: When using DC as the InfiniBand transport type to perform an ib_read RDMA operation between ConnectX-7 (NDR) and ConnectX-8, a bandwidth degradation of approximately 25% may be observed when using a low number of QPs (1–16). The performance degradation diminishes as the number of QPs increases.

Workaround: N/A

Keywords: DC, ib_read RDMA, NDR, performance

Detected in version:

40.47.1026

4685736

Description: Creating a DPA process that allocates a 128 MB data segment and loads a dynamic library may fail with syndrome 0xdc30ac.

Workaround: Limit the DPA application’s data segment size to 64 MB.

Keywords: DPA

Detected in version:

40.47.1026

4618452

Description: When only a partial number of fiber lanes are connected to a module (i.e., not all supported lanes are populated), the module re-insertion process may experience an extended link-up time of approximately one minute. This occurs because the module requires additional time to detect that only a subset of the lanes is connected.

Workaround: N/A

Keywords: Cables

Detected in version:

40.47.1026

4270913

Description: For DC InfiniBand transport, using SRQs requires distributing QPs across four SRQs. Each QP must be assigned to its own SRQ when there are up to four QPs.

Workaround: If more than four QPs are used, they must be evenly distributed across the four available SRQs.

Keywords: DC InfiniBand transport, SRQ, QPs

Detected in version:

40.47.1026

4665802

Description: Due to a re-burst scheduler limitation, the expected “speed of light” value for single QP RDMA Write on ConnectX-8 devices cannot be calculated. Currently, verification is limited to identifying performance degradations relative to previous firmware versions.

Workaround: N/A

Keywords: Single QP RDMA Write

Detected in version:

40.47.1026

4412310

Description: Split operation (port splitting) on the second port is not supported on the ConnectX-8 SuperNIC model 900-9X81Q-00CN-ST0.

Workaround: N/A

Keywords: Port splitting

Detected in version:

40.45.1020

4394475

Description: The existing congestion control configuration applies globally, rather than on a per-priority basis.

Workaround: Ensure that the configuration values for all priorities are aligned in either mlxconfig ROCE_CC_PRIO_MASK_P$port or sysfs ecn/roce_rp/enable/$port.

Keywords: Congestion control, ROCE_CC_PRIO

Detected in version:

40.45.1020

4240828

Description: Simultaneous access to the same PDDR module info page is not supported.

Workaround: N/A

Keywords: PDDR module

Detected in version:

40.45.1020

4230775

Description: Due to a known issue, telemetry rate must be set to lower than 3 minutes.

Workaround: N/A

Keywords: Telemetry rate

Detected in version:

40.44.0212

4038325 / 
4031430 / 
4038341 / 
4046105

Description: Connecting to systems with NRZ speeds of 1,10, 25, 40, 50, or 100Gb/s is not supported.

Workaround: N/A

Keywords: NRZ, Connectivity

Detected in version:

40.44.0208

4201405

Description: Upgrading to firmware 40.44.0xxx from any previous Engineering Sample version requires power cycling the driver and not just resetting it (using mlxfwreset).  

Workaround: N/A

Keywords: Upgrade, power cycle, reset

Detected in version:

40.44.0208

Last updated: