NVIDIA MLNX-GW User Manual for NVIDIA Skyway Appliance

MetroX XC Routing Overview

InfiniBand Interfaces

show interfaces ib 


show interfaces ib <interface>

Displays information on InfiniBand interfaces.

Syntax Description

Interface

Device/port (e.g., 1/1)

Default

N/A

Configuration Mode

Any command mode

History

18.01.5000

18.02.1000

Example

metrox (config) # show interfaces ib 1/1 IB1/1: Admin state        : Enabled  Logical port state : LinkDown  Physical port state: Polling  GUID               : e8eb:d303:0066:55fe  Supported speeds   : sdr fdr edr  Speed              : N/A  Supported widths   : 1x 2x 4x  Max supported MTU  : 4096  MTU                : 4096  VL capabilities    : VL0-VL3  Description        :   IB Subnet          : InfiniBand-default  Phy-profile        : high-speed-ber  Rx:    packets      : 0    bytes        : 0    error packets: 0  Tx:    packets      : 0    bytes        : 0    error packets: 0

Related Commands

show interfaces long-haul

Notes

If the interface is not specified, then the command prints all the InfiniBand interfaces.

show interfaces ib brief 


show interfaces ib brief

Displays brief information on the InfiniBand interfaces.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

18.01.5000

18.02.1000

Example

metrox (config) # show interfaces ib brief -------------------------------------------------------------Name        Admin state   Logical Port state        Speed     -------------------------------------------------------------IB1/1       Enabled       LinkActive              edr       IB2/1       Enabled       LinkActive              edr  

Related Commands

show interfaces ib

Notes


interface ib shutdown


interface ib <interface> shutdown
no interface ib <interface> shutdown

Disables/Enables InfiniBand physical interface.

Syntax Description

Interface

Device/port (e.g., 1/1)

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface ib 1/1 shutdown

Related Commands

show interfaces ib

Notes


interface ib description


interface ib <interface>  description
no interface ib <interface> description

Adds a description for an interface.
The no form of the command removes a description for an interface.

Syntax Description

Interface

Device/port (e.g., 1/1)

Description

A free-form string that has no semantics other than being displayed when the interface records are listed

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface ib 1/1 description first port

Related Commands

show interfaces ib

Notes


Long-Haul Interfaces

show interfaces long-haul


show interfaces long-haul <interface>

Displays information on long-haul interfaces.
If the interface is not specified, then the command prints all the InfiniBand
interfaces.

Syntax Description

Interface

Device/port (e.g., 1/1)

Default

N/A

Configuration Mode

Any command mode

History

18.01.5000

Example

metrox (config) # show interfaces long-haul 1/2 LH1/2:  Admin state             : Enabled  Operational state       : Up  Description             :   MAC address             : E8:EB:D3:66:55:FF  MTU                     : 9216  FEC                     : auto  Supported speeds        : 100G  Actual speed            : Unknown  Auto-negotiation        : Enabled  Actual latency (ms)     : 0  Last latency measurement: N/A  Remote KA status        : Failure  Last success KA message : N/A  IPv4 address:    N/A  Remote IPv4 address:    N/A  Rx:    packets      : 299    bytes        : 20830    error packets: 0  Tx:    packets      : 14    bytes        : 1116   error packets: 0

Related Commands

show interfaces ib

Notes


show interfaces long-haul brief


show interfaces long-haul brief

Displays brief information on the long-haul interfaces.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

18.01.5000

Example

metrox (config) # show interfaces long-haul brief  -------------------------------------------------------------Name        Admin state   Oper state              Speed     -------------------------------------------------------------LH1/2       Enabled       Up                      100G      LH2/2       Enabled       Up                      100G  

Related Commands

show interfaces long-haul

Notes


interface long-haul shutdown

New command

interface long-haul <interface> shutdown
no interface long-haul <interface> shutdown

Disables/Enables long-haul interface.

Syntax Description

Interface

Device/port (e.g., 1/2)

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface long-haul 1/2 shutdown

Related Commands

show interfaces long-haul

Notes


interface long-haul description


interface long-haul <interface> description
no interface long-haul <interface> description

Adds a description for an interface.
The no form of the command removes a description for an interface.

Syntax Description

Interface

Device/port (e.g., 1/2)

Description

A free-form string that has no semantics other than being displayed
when the interface records are listed

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface long-haul 1/2 description first port

Related Commands

show interfaces long-haul
interface long-haul remote ip address

Notes

Both the local and remote IP addresses should be configured in order to pass traffic.

interface long-haul remote ip address


interface long-haul <interface> remote ip address <ip-address>
no interface long-haul <interface> remote ip address <ip-address> [<ip address>]

Enters user-defined remote IPv4 address for the interface.
The no form of the command removes the specified remote IPv4 address. If no address is
specified, then all IPv4 addresses of this interface are removed.

Syntax Description

interface

Device/port (e.g., 1/2)

ip-address

IPv4 address

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface long-haul 1/2 remote ip address 1.1.1.1/24

Related Commands

show interfaces long-haul
interface long-haul ip address

Notes


interface long-haul ip address


interface long-haul <interface> ip address <ip-address> <mask>
no interface long-haul <interface> ip address <ip-address> [<ip address>]

Enters user-defined IPv4 address for the interface.
The no form of the command removes the specified IPv4 address. If no address is
specified, then all IPv4 addresses of this interface are removed.

Syntax Description

interface

Device/port (e.g., 1/2)

ip-address

IPv4 address

mask

There are two possible ways to the mask:/length (i.e. /24)
Network address (i.e. 255.255.255.0) The mask length may be configured without a space (i.e. <ipv4-address>/<length>)

Default

N/A

Configuration Mode

config

History

18.01.5000

Example

metrox (config) # interface long-haul 1/2 ip address 1.1.1.1/24

Related Commands

show interfaces long-haul
interface long-haul ip address

Notes

Both the local and remote IP addresses should be configured in order to pass traffic.

Routing and ECMP

ip route


ip route <ip-prefix> <netmask> {<next -hop-ip-address> | ethernet <port>} [<distance>]
no ip route <ip-prefix> <netmask> [<next -hop-ip-address>]

Configures a static route.
The no form of the command removes the static route configured.

Syntax Description





ip-prefix

IP address.

netmask

There are two possible ways to input the mask:

  • /<length> (e.g., /24)

  • Network address (e.g., 255.255.255.0)

next-hop-ip-address

IP address of the next hop

Default

N/A

Configuration Mode

config

History

8.0.0100

Example

metrox (config) # ip route 80.80.80.0 /24 20.20.20.2

Related Commands

show ip route

Notes

The route is configurable between the Ethernet and InfiniBand subnets

show ip route


show ip route [static] 

Displays routing table.

Syntax Description

static

Displays entries added through CLI commands.

Configuration Mode

config

History

18.01.5002

Example

metrox (config) # show ip route Flags: F: Failed to install in H/W B: BFD protected (static route) i: BFD session initializing (static route) x: protecting BFD session failed (static route) c: consistent hashing p: partial programming in H/W VRF Name default: ----------------------------------------------------------------------------------- Destination Mask Flag Gateway Interface Source AD/M ----------------------------------------------------------------------------------- default 0.0.0.0 10.12.67.126 mgmt0 DHCP 1/1 10.12.67.0 255.255.255.128 0.0.0.0 mgmt0 direct 0/0 192.168.2.0 255.255.255.0 c 0.0.0.0 vlan1 direct 0/0

Related Commands

ip route

Notes

  • If no default route exists, then the message “Route not found” is printed

  • Route next hop is BFD controlled, status is viewable when <all> is inserted in the command, and it will be shown as follows:

    • If route is removed from routing decision it will be marked as “Active”

    • Protected next hops are marked with “B”

    • BFD protected failed/non active neighbors are marked with “BF”


IP Diagnostic Tools

ping


ping <interface> [-LRUbdfnqrvVaA] [-c count] [-i interval] [-w deadline] [-p pattern] [-s packetsize] [-t ttl] [-I interface or address] [-M mtu discovery hint] [-S sndbuf] [-T timestamp option ] [-Q tos ] [hop1 ...] destination

Sends ICMP echo requests to a specified host.

Syntax Description

Interface

Device/port (e.g., 1/2)

Linux Ping options

Default

N/A

Configuration Mode

config

History

18.01.5002

Example

metrox (config) # ping interface long-haul 1/2 1.1.1.1PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=0.456 ms64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=0.222 ms^C--- 1.1.1.1 ping statistics ---2 packets transmitted, 2 received, 0% packet loss, time 1038msrtt min/avg/max/mdev = 0.222/0.339/0.456/0.117 ms

Related Commands

traceroute

Notes

When using -I option use the interface name + interface number, for example “ping -I vlan10”

traceroute


traceroute <interface>[-46dFITUnrAV] [-f first_ttl] [-g gate,...] [-i device] [-m max_ttl] [-N squeries] [-p port] [-t tos] [-l flow_label] [-w waittime] [-q nqueries] [-s src_addr] [-z sendwait] host [packetlen] 

Traces the route packets take to a destination.

Syntax Description

Interface

Device/port (e.g., 1/2)

-4

Uses IPv4

-d

Enables socket level debugging

-F

Sets DF (“do not fragment” bit) on

-I

Uses ICMP ECHO for tracerouting

-T

Uses TCP SYN for tracerouting

-U

Uses UDP datagram (default) for tracerouting

-n

Does not resolve IP addresses to their domain names

-r

Bypasses the normal routing and send directly to a host on an attached network

-A

Performs AS path lookups in routing registries and print results directly after the corresponding addresses

-V

Prints version info and exit

-f

Starts from the first_ttl hop (instead from 1)

-g

Routes packets throw the specified gateway (maximum 8 for IPv4 and 127 for IPv6)

-i

Specifies a network interface to operate with

-m

Sets the max number of hops (max TTL to be reached)
Default: 30

-N

Sets the number of probes to be tried simultaneously
Default: 16

-p

Uses destination port. It is an initial value for the UDP destination port (incremented by each probe, default is 33434), for the ICMP seq number (incremented as well, default from 1), and the constant destination port for TCP tries (default is 80).

-t

Sets the TOS (IPv4 type of service) or TC (IPv6 traffic class) value for outgoing packets

-l

Uses specified flow_label for IPv6 packets

-w

Sets the number of seconds to wait for response to a probe (default is 5.0). Non-integer (float point) values allowed too.

-q

Sets the number of probes per each hop
Default: 3

-s

Uses source src_addr for outgoing packets

-z

Sets minimal time interval between probes (default is 0). If the value is more than 10, then it specifies a number in milliseconds, else it is a number of seconds (float point values allowed too).

Default

N/A

Configuration Mode

config

History

18.01.5002

Example

metrox (config) # traceroute interface long-haul 1/2 192.168.10.70 traceroute to 192.168.10.70 (192.168.10.70), 30 hops max, 40 byte packets 1 172.30.0.1 (172.30.0.1) 3.632 ms 2.849 ms 3.544 ms 2 10.222.128.46 (10.222.128.46) 3.176 ms 3.289 ms 3.656 ms 3 10.158.128.30 (10.158.128.30) 15.331 ms 15.819 ms 16.388 ms 4 10.158.128.65 (10.158.128.65) 20.468 ms 7.893 ms 12.27 ms 5 10.7.34.115 (10.7.34.115) 16.405 ms 11.985 ms 12.264 ms 6 192.168.10.70 (192.168.10.70) 16.377 ms 16.091 ms 20.475 ms

Related Commands


Notes

  • The following flags are not supported: -6, -l, -A

  • When using -i option use the interface name + interface number, for example “traceroute -i mgmt0”

tcpdump


tcpdump <interface>[-aAdeflLnNOpqRStuUvxX] [-c count] [-C file_size] [-E algo:secret] [-F file] [-i interface] [-M secret] [-r file] [-s snaplen] [-T type] [-w file] [-W filecount] [-y datalinktype] [-Z user] [expression]

Invokes standard binary, passing command line parameters straight through. Runs in foreground, printing packets as they arrive, until the user hits Ctrl+C.

Syntax Description

Interface

Device/port (e.g., 1/2)

Linux Ping options

Default

N/A

Configuration Mode

config

History

18.01.5002

Example

metrox (config) # tcpdump interface long-haul 1/2...... 09:37:38.678812 IP 192.168.10.7.ssh > 192.168.10.1.54155: P 1494624:1494800(176) ack 625 win 90 <nop,nop,timestamp 5842763 858672398> 09:37:38.678860 IP 192.168.10.7.ssh > 192.168.10.1.54155: P 1494800:1495104(304) ack 625 win 90 <nop,nop,timestamp 5842763 858672398> ... 9141 packets captured 9142 packets received by filter 0 packets dropped by kernel

Related Commands


Notes

  • When using -i option use the interface name + interface number, for example “tcpdump -i mgmt0”

  • For all flag options of this command refer to the linux ‘man page’ of tcp dump

System Commands

show system state


show system state

Displays information on the state of the ConnectX cards (adapter cards) of the system.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

18.01.5002

Example

metrox (config) # show system stateSystem state            : system-readySystem state description: system is ready to send/receive traffic------------------------------------------------------------------------Card ID          Ports              Card type          Card state       ------------------------------------------------------------------------1                IB 1/1, LH 1/2     ConnectX7          card-ready2                IB 2/1, LH 2/2     ConnectX7          card-readyIB-MGMT          IB-MGMT1           ConnectX7          card-ready

Related Commands



Notes


show asic-version


show asic-version

Displays FW versions of the ConnectX7 cards installed on the system

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

18.01.5002

Example

metrox (config) # show asic-version-----------------------------------------------------Device           Type               Version          -----------------------------------------------------1                ConnectX7          28.0.99.12902                ConnectX7          28.0.99.1290IB-MGMT          ConnectX7          28.0.99.1290

Related Commands


Notes


Last updated: