NVIDIA DOCA Comm Channel Admin Tool

This document describes the Comm Channel Admin Tool, used to monitor Comm Channel services, connections, etc.

Introduction

Comm Channel Admin Tool is used to monitor Comm Channel services and connections on both the DPU and the host. For more information on the DOCA Comm Channel library, refer to the DOCA Comm Channel – Deprecated.

Prerequisites

NVIDIA® BlueField®-2 firmware version 24.35.1012 or higher.

Description

On the DPU, Comm Channel Admin Tool can show the user which services are up alongside additional information about them:

service_name – the name of the service
transport_type – UD or DC
allowed_vhca_id – which vhca_id is allowed to connect to this service
service_qpn_or_dct –
- UD transport type – indicates QP number of the service
- DC transport type – indicates DC number of the service
num_connected_client – indicates the number of current connected clients
max_num_connected_client – indicates the maximum number of connected clients

Furthermore, users may query specific services according to the service name to get information about the connected clients. For every connected client the tool shows:

connection_id – unique identifier for the connection
qpn – QP number (or DCT) used for the connection

On the host, the tool can show the user the name of the active services on the DPU. Furthermore, it can show information about the active connections between the host and the DPU:

service_name – the name of the service
transport_type – UD or DC
my_qpn_or_dci –
- UD transport type – indicates QPN of the connection initiator
- DC transport type – the field indicates the DCI of the connection initiator
connection_type – CONNECT_BY_SERVICE_ID or CONNECT_BY_VHCA_ID
dst_vhca_id – vhca_id of the target side of the connection. Valid only when the connection type is CONNECT_BY_VHCA_ID.

Execution

To execute the DOCA Comm Channel Admin Tool, run:

/opt/mellanox/doca/tools/doca_comm_channel_admin_tool

Afterwards, users get an interactive CI that awaits a command:

Comm-Channel-Admin-Tool >>

The commands and their flags can be obtained by writing help:

Comm-Channel-Admin-Tool >> help
Comm-Channel-Admin-Tool CLI Commands:
     service, s          To be used on the DPU
         --all, -a                               Show all services
         --service-name, -sn [service_name]      Show specific service and its connected clients
     connection, c       To be used on the host
         --all, -a                               Show all connections
         --service-name, -sn [service_name]      Show specific connection according to service name

     discovery, d        To be used on the host
         --all, -a                               Show all active services
         --service-name, -sn [service_name]      Show specific active service according to service name

     quit            Exit tool

On the DPU, to see all services, users may use the service (or s) command with the flag --all (or -a). For example:

Comm-Channel-Admin-Tool >> service --all
2022-09-28 09:54:28,410 - Comm-Channel-Admin-Tool - INFO - On /dev/mst/mt41686_pciconf0:
Active services:
+-----------------------+------------------+-------------------+----------------------+------------------------+----------------------------+
| service_name          | transport_type   |   allowed_vhca_id |   service_qpn_or_dct |   num_connected_client |   max_num_connected_client |
+=======================+==================+===================+======================+========================+============================+
| secure_channel_server | DC               |                 0 |                 4548 |                      1 |                        512 |
+-----------------------+------------------+-------------------+----------------------+------------------------+----------------------------+
2022-09-28 09:54:28,886 - Comm-Channel-Admin-Tool - INFO - On /dev/mst/mt41686_pciconf0.1:
No active services

To query a specific service and see its connected clients, users may use the --serivce-name (or -sn) flag while providing the service_name. For example:

Comm-Channel-Admin-Tool >> service --service-name secure_channel_server
2022-09-28 09:56:16,335 - Comm-Channel-Admin-Tool - INFO - On /dev/mst/mt41686_pciconf0:
+-----------------------+------------------+-------------------+----------------------+------------------------+----------------------------+
| service_name          | transport_type   |   allowed_vhca_id |   service_qpn_or_dct |   num_connected_client |   max_num_connected_client |
+=======================+==================+===================+======================+========================+============================+
| secure_channel_server | DC               |                 0 |                 4548 |                      1 |                        512 |
+-----------------------+------------------+-------------------+----------------------+------------------------+----------------------------+ 

Connected clients:
+-----------------+------------------+
|  connection_id  | transport_type   |
+=================+==================+
|               0 |             4547 |
+-----------------+------------------+ 
2022-09-28 09:56:16,809 - Comm-Channel-Admin-Tool - INFO - On /dev/mst/mt41686_pciconf0.1:
No active service by name secure_channel_server

On the host, to see all active services, users may use the discovery (or d) command with --all flag (or -a). For example:

Comm-Channel-Admin-Tool >> discovery --all
2022-09-28 12:58:42,201 - Comm-Channel-Admin-Tool - INFO - On /dev/mst/mt41686_pciconf0:
Active services:
+-----------------------+
| service_name          |
+=======================+
| secure_channel_server |
+-----------------------+
2022-09-28 12:58:42,632 - Comm-Channel-Admin-Tool - INFO - On /dev/mst/mt41686_pciconf0.1:
No active services

Users may also filter to show only specific services according to the service_name by using the --serivce-name (or -sn) flag while providing the service_name.

On the host, to see all active connections, users may use the connection command (or c) with --all flag (or -a). For example:

Comm-Channel-Admin-Tool >> connection --all
2022-09-28 13:01:54,420 - Comm-Channel-Admin-Tool - INFO - On /dev/mst/mt41686_pciconf0:
Active connections:
+-----------------------+------------------+-----------------+-----------------------+---------------+
| service_name          | transport_type   |   my_qpn_or_dci | connection_type       |   dst_vhca_id |
+=======================+==================+=================+=======================+===============+
| secure_channel_server | DC               |              71 | CONNECT_BY_SERVICE_ID |             0 |
+-----------------------+------------------+-----------------+-----------------------+---------------+ 
2022-09-28 13:01:54,860 - Comm-Channel-Admin-Tool - INFO - On /dev/mst/mt41686_pciconf0.1:
No active Connections

Users may also filter to show only specific connections according to service_name by using the --serivce-name (or -sn) flag while providing the service_name.

If a new service or connection is added, there is no need to restart the tool. After each command, the tool always shows the latest services and connections.

Last updated: August 14, 2024