Introduction

PCI Express Slot

In PCIe x16 Configuration
PCIe Gen6 @ 64GT/s through x16 edge connector

System Power Supply

Refer to Specifications

Operating System

• In-box drivers for major operating systems:

  • Linux: RHEL, Ubuntu

  • Windows

• DOCA Host

• OpenFabrics Windows Distribution (WinOF-2)

Connectivity

• Interoperable with up to 800 Gb/s Ethernet switches and up to XDR InfiniBand switches

• Passive copper cable with ESD protection

• Powered connectors for optical and active cable support

1

ConnectX-9 SuperNIC for OCP 3.0

1

Internal Lock Bracket

According to the OPN you have purchased, the card uses the following PCIe express interfaces:

• PCIe x16 configurations:
  PCIe Gen6 (64GT/s) through x16 edge connector

• 2x PCIe x8 configurations (Socket-Direct):
  PCIe Gen6 ( SERDES @ 64GT/s / 32GT/s) through x16 edge connector

ConnectX-9 delivers low latency, high bandwidth, and computing efficiency for high-performance computing (HPC), artificial intelligence (AI), and hyperscale cloud data center applications. ConnectX-9 is InfiniBand Architecture Specification v1.7 compliant.

InfiniBand Network Protocols and Rates

With these basic rates, 800Gb/s can be built from 4 ports of 200Gb/s, for example.

ConnectX-9 SuperNICs comply with the following IEEE 802.3 standards:
800GbE / 400GbE / 200GbE / 100GbE / 25GbE / 10GbE

• SPI - includes 512Mbit SPI Quad Flash device.

• FRU EEPROM - Stores the parameters and personality of the SuperNIC. The EEPROM capacity is 128Kbit. FRU I2C address is (0x50) and is accessible through the PCIe SMBus. (Note: Address 0x58 is reserved.)

In order to better scale their networks, data center operators often create overlay networks that carry traffic from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN. While this solves network scalability issues, it hides the TCP packet from the hardware offloading engines, placing higher loads on the host CPU. ConnectX-9 effectively addresses this by providing advanced NVGRE and VXLAN hardware offloading engines that encapsulate and de-capsulate the overlay protocol.

Support for port-based Quality of Service enabling various application requirements for latency and SLA.

ConnectX-9 provides dedicated adapter resources and guaranteed isolation and protection for virtual machines within the server.

ConnectX-9 SR-IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines (VM) within the server.

• Vector collective operations offload

• MPI_Alltoall offloads

• Rendezvous protocol offload

The secure boot process assures the booting of authentic firmware/software that is intended to run on ConnectX-9. This is achieved using a hybrid approach consisting of both post-quantum and traditional cryptography. ConnectX-9 supports several cryptographic functions in its HW Root-of-Trust (RoT) that has its key stored in on-chip fuses.

The Secure firmware update feature enables a device to verify digital signatures of new firmware binaries using a hybrid approach consisting of both post-quantum and traditional cryptography to ensure that only officially approved versions can be installed from the host, the network, or a Board Management Controller (BMC). The firmware of devices with “secure firmware update” functionality (secure FW), restricts access to specific commands and registers that can be used to modify the firmware binary image on the flash, as well as commands that can jeopardize security in general.

For further information, refer to the MFT User Manual.

TBD

ConnectX-9 technology maintains support for host manageability through a BMC. ConnectX-9 PCIe stand-up adapter can be connected to a BMC using MCTP over SMBus or MCTP over PCIe protocols as if it is a standard NVIDIA PCIe stand-up SuperNIC. For configuring the adapter for the specific manageability solution in use by the server, please contact NVIDIA Support.

• Protocols: PLDM, NCSI

• Transport layer – RBT, MCTP over SMBus, and MCTP over PCIe

• Physical layer: SMBus 2.0 / I2C interface for device control and configuration, PCIe

• PLDM for Monitor and Control DSP0248 

• PLDM for Firmware Update DSP026 

• IEEE 1149.6 

• Secured FW update  

• FW Recovery  

• NIC reset 

• Monitoring and control 

• Network port settings 

• Boot setting

ConnectX-9, utilizing IBTA RDMA (Remote Data Memory Access) and RoCE (RDMA over Converged Ethernet) technology, delivers low latency and high-performance over InfiniBand and Ethernet networks. Leveraging datacenter bridging (DCB) capabilities as well as ConnectX-9 advanced congestion control hardware mechanisms, RoCE provides efficient low-latency RDMA services over Layer 2 and Layer 3 networks.

PeerDirect™ communication provides high-efficiency RDMA access by eliminating unnecessary internal data copies between components on the PCIe bus (for example, from GPU to CPU), and therefore significantly reduces application run time. ConnectX-9 advanced acceleration technology enables higher cluster efficiency and scalability to tens of thousands of nodes.

Adapter functionality enables reduced CPU overhead allowing more available CPU for computation tasks.

• Flexible match-action flow tables

• Open VSwitch (OVS) offload using ASAP²®

• Tunneling encapsulation/decapsulation

ConnectX-9 supports IPsec, PSP cryptography, and TLS/XLS acceleration. Its SuperNIC hardware-based accelerations offload cryptographic operations from the CPU, reducing latency and enabling scalable, high-performance crypto solutions.

• RDMA and RoCEv2 accelerations

• Advanced, programmable congestion control

• NVIDIA GPUDirect® RDMA

• GPUDirect Storage

• In-Network Computing

• High-speed packet reordering

• MPI accelerations