NVIDIA Q32xx and Q34xx XDR 800Gb/s InfiniBand Switch Systems User Manual

Interfaces

The systems support the following interfaces:

  • Data interfaces - InfiniBand

  • 10/100/1000Mb Ethernet management interface (RJ45)

  • USB port (2.0 type A)

  • RS232 Console port (RJ45)

  • I²C interface*

  • FNM - Fabric Network Management Port 

  • Reset button

  • Status and Port LEDs

In order to review the full configuration options matrix, refer to Management Interfaces, PSUs and Fans.

Data Interfaces

The full list of interfaces per system is provided in Speed and Switching Capabilities.

Speed

InfiniBand speed is auto-adjusted by the InfiniBand protocol. NVIDIA systems support QDR/FDR/EDR/HDR/NDR/XDR InfiniBand.

  • FDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 14.0625Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 56.25Gb/s.

  • EDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 25Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 100Gb/s.

  • HDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 50Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 200Gb/s.

  • NDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 100Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 400Gb/s.

  • XDR is an InfiniBand data rate, where each lane of a 4X port runs a bit rate of 200Gb/s with 64b/66b encoding, resulting in an effective bandwidth of 800Gb/s.

RS232 (Console)

The RS232 serial “Console” port is labeled image2019-3-5_16-50-5.png .

The Console port is an RS232 serial port on the front side of the chassis that is used for initial configuration and debugging. Upon first installation of the system, you need to connect a PC to this interface and configure network parameters for remote connections. Refer to Configuring Network Attributes to view the full procedure.

This interface is not found in externally managed systems.

Management

The RJ45 Ethernet “MGT” port is labeled image2019-3-5_16-52-4.png .

The Management RJ45 Ethernet ports provide access for remote management. The management ports are configured with auto-negotiation capabilities by default (100MbE to 1GbE). The management ports’ network attributes (such as IP Address) need to be pre-configured via the RS232 serial console port or by DHCP before use. Refer to Configuring Network Attributes to view the full procedure.

This interface is not found in externally managed systems.

USB

The USB interface is USB 2.0 type A compliant and can be used by the operating system to connect to an external disk for software upgrade or file management.

  • USB 1.0 is not supported.

  • Do not use excessive force when inserting or extracting the USB disk to and from the connector.

  • This interface is not found in externally managed systems.

I²C

The I²C connector is combined with the USB connector, and is located on the front side of the system. It can be used with the I²C DB9 to micro USB splitting harness.

  • This interface is not found in managed systems.  

  • Apart from the initial configuration, I²C interface is made exclusively for debugging and troubleshooting. Only FAEs are authorized to connect through it.

FNM - Fabric Network Management Port

The FNM port is a separate OSFP InfiniBand in-band management port. It enables accessing the UFM platform that allows data center operators to efficiently monitor and operate the entire fabric. The following table describes the supported transceivers for the Q3450 planarized FNM port.. For more information on the Q3200 and Q3400 clusters, please refer to the NVIDIA UFM Enterprise Appliance Hardware User Manual.

Q3450 Photonics  - Planarized FNM Port  

Mode

Description

Transceivers Legacy Part Number

Transceivers SKU

NDR SM - Single Mode  

FNM side 

1 port is active in 400Gb/s, switch side (Finn) 

 MMS4X00-NS (100m) 

980-9I30H-00NM00 

NDR SM - Single Mode 

UFM 3.5 side 

1 port in 400Gb/s (RHS) 

 MMS4X00-NS400 (100m) 

980-9I31N-00NM00  

 NDR MM 

FNM side 

1 port is active in 400Gb/s, switch side (Finn) 

 MMA4Z00-NS (50m) 

980-9I510-00NS00 

 NDR MM 

UFM 3.5 side 

1 port in 400Gb/s (RHS) 

 MMA4Z00-NS400 (50m) 

980-9I51S-00NS00 

Reset Button

The reset button is located on the front side of the system. It is a red button that requires a tool to be pressed.

Do not use a sharp pointed object such as a needle or a push pin for pressing the reset button. Use a flat object to push the reset button.

  • To reset the system, push the reset button for less than 15 seconds.

  • When using an NVOS based system, keeping the reset button pressed for more than 15 seconds will reset the system and the “admin” password, this should allow you to enter without a password and set a new password for the user “admin”.

LEDs

See LED Notifications.

Last updated: