DOCA SDK Documentation

DPL Service Configuration

1. Introduction

The DPL Runtime Service utilizes three distinct types of configuration files. These files adhere to a simplified INI-style format, which supports repeated sections (e.g., [INTERFACE]) and uses # for comments rather than ;.

The available configuration files are as follows:

  • DPL Device Configuration: Located at /etc/dpl_rt_service/devices.d/<device-id>.conf.

  • Service Daemon Configuration: Located at /etc/dpl_rt_service/dpl_rt.conf.

  • System Configuration: Located at /etc/dpl_rt_service/system.conf.

The dpl_dpu_setup.sh script installs default (recommended) daemon and system configuration files. However, users must create their own DPL Device configuration file based on the provided template, which is also installed by the setup script. 

Any changes made to files within the /etc/dpl_rt_service/ directory require a restart of the DPL Runtime Service Container to take effect. Please refer to the section "Restarting the DPL Runtime Service After Configuration Changes" for instructions.

2. DPL Device Configuration

Path: /etc/dpl_rt_service/devices.d/<device-id>.conf (e.g., /etc/dpl_rt_service/devices.d/1000.conf).

This file defines the device, its interfaces, and the mapping to DPL Port IDs used in DPL programs. A configuration template is available at /etc/dpl_rt_service/devices.d/NAME.conf.template

When using SFs or SR-IOV VFs, ensure the configuration references their representor interfaces, not the SF/VF interfaces themselves.

2.1. DPL Port ID Mapping

Each physical or virtual interface is assigned a logical DPL Port ID (dpl_logical_port_id). Consistency between the DPL program and this configuration is mandatory.

Supported interface types include:

  • Uplink netdev interface (e.g., p0).

  • PF representor (e.g., pf0hpf).

  • VF representor (e.g., pf0vf0).

  • SF representor (e.g., en3f0pf0sf1).

Unless multiport eswitch is enabled, all listed interfaces must belong to the same uplink port.

2.2. DPL Port ID Rules

The following rules apply to dpl_device_id and dpl_logical_port_id values:

Condition

Requirement

Reserved value

UINT32_MAX

DPL Device ID

Must be a positive integer

DPL Interface ID

Integer between 0 and UINT32_MAX

Uplink ports per config

Exactly one (unless multiport eswitch is enabled)

2.3. Example DPL Device Configuration File

The following is an example of a valid device configuration file.

Bash
# [DEVICE] section: Must appear only once.
[DEVICE]
# The DPL Device ID used by the controller to manage tables.
dpl_device_id=1000
# Cache counter timeout to decrease HW accesses.
dpl_counter_cache_timeout=0
# Counter polling interval for idle-timeout (in seconds).
idle_timeout_polling_interval=2
# Polling interval for TCP state objects from HW (in milliseconds).
tcp_state_polling_interval_msec=200

# [P4_RT_CONTROLLER] section: Must appear only once.
[P4_RT_CONTROLLER]
# DPL Port ID for traffic originating from the controller.
p4_controller_port_id=9876

# [INTERFACE] section: Repeated for each DPL Port (network interface).
[INTERFACE]
interface=p0
dpl_logical_port_id=0
mtu=1514
# mac=00:00:00:00:00:00

[INTERFACE]
interface=pf0hpf
dpl_logical_port_id=65535
mtu=1514

[INTERFACE]
interface=pf0vf0
dpl_logical_port_id=1
mtu=1514

2.4. Section DEVICE

This section defines the attributes of the DPL Device, which consists of multiple ports and supports DPL program loading. It must appear exactly once per device .conf file.

  • dpl_device_id The Device ID used by controller applications to identify the target device for connection and program management.

  • dpl_counter_cache_timeout This parameter reduces hardware accesses for counter reads. When the cache expires, a hardware access occurs upon the next request. Note that enabling this may result in the retrieval of outdated counter values.

  • idle_timeout_polling_interval Sets the polling interval (in seconds) for reading entry counters from hardware and identifying stale entries (those with no traffic for a period exceeding the defined threshold). This interval affects both Entry Timeout and Delayed Counter Statistics. Larger values impact the accuracy of timeout notifications and the refresh rate of counter statistics.

  • tcp_state_polling_interval_msec Defines the interval (in milliseconds) for reading TCP state objects from the hardware.

2.5. Section P4_RT_CONTROLLER

This section configures communication with P4 Controllers.

p4_controller_port_id P4 controller applications can send packets ("Packet Out") to the DPL Runtime Service via RPC messages. These packets are assigned a port ID equal to p4_controller_port_id. A DPL programmer can identify traffic originating from a P4 Controller by comparing std_meta.ingress_port to this value.

2.6. Section INTERFACE

This section defines the attributes of a port belonging to the device. This section may be repeated for multiple interfaces.

  • interface: The net-device interface name on the system.

  • dpl_logical_port_id: The ID used to reference the port in DPL programs and table updates. It is used for matching ingress traffic and directing egress traffic. Programmers can verify the source port by comparing std_meta.ingress_port to this value.

  • mtu: Defines the Ethernet frame size (MTU).

    Currently, this field is ignored.

  • mac: Defines the MAC address.

    Currently, this field is ignored.

3. Service Daemon Configuration

Path: /etc/dpl_rt_service/dpl_rt.conf.

This file configures core behaviors for the DPL Runtime Service, including logging and gRPC server binding addresses.

3.1. Section LOGGING

  • log_file_path: Defines the path to the dpl_rtd log file.

  • log_level: Sets the default log level upon startup. Supported values (case-insensitive) include: DISABLE, CRITICAL, ERROR, WARNING, INFO, DEBUG, and TRACE

    Log levels can be changed dynamically via the DPL Admin client but will not persist across restarts unless updated in this file.

3.2. RPC Server Sections

The daemon supports three distinct gRPC servers:

  1. [P4RT_RPC_SERVER]: Listens for P4Runtime client connections. A sample open-source client is included in the DPL Dev container.

  2. [DPL_ADMIN_RPC_SERVER]: Listens for dpl_admin tool connections.

  3. [DPL_NSPECT_RPC_SERVER]: Listens for DPL Debugger tool connections.

3.2.1. Common gRPC Server Parameters

The following parameters apply to all three server sections:

  • server_address: The TCP binding address. Use [::] (IPv6 ANY) to allow connections from all interfaces (including IPv4), or specify a specific IP address.

  • server_tcp_port: The TCP binding port. If a non-default port is configured, client applications must specify this port when connecting.

3.2.2. TLS Authentication

TLS authentication can be enabled independently for each server. To enable TLS, the following mutually inclusive parameters must be added to the relevant server section:

  • server_cert: Path to the server.crt file.

  • ca_cert: Path to the ca.crt file.

  • server_key: Path to the server.key file.

Make sure to provide the client side with the corresponding parameters.

Example TLS Setup
Bash
# OpenSSL PKI Setup (Root CA, Server & Client Certificates)

# This guide creates a simple self-signed PKI:
# - Root CA
# - Server certificate 
# - Client certificate 

password=abcd

## Establish a Private Root CA
### 1. Generate CA Private Key
openssl genrsa -passout pass:$password -des3 -out ca.key 4096

### 2. Generate Self-Signed Root Certificate
openssl req -passin pass:$password -new -x509 -days 365 -key ca.key -out ca.crt \
  -subj "/C=US/ST=CA/L=PaloAlto/O=Test/OU=Test/CN=Root CA"

## Generate the Server Certificate

### 1. Create Server key
openssl genrsa -passout pass:$password -des3 -out server.key 4096

### 2. Create Certificate Signing Request (CSR)
openssl req -passin pass:$password -new -key server.key -out server.csr \
  -subj "/C=US/ST=CA/L=PaloAlto/O=Test/OU=Server/CN=localhost"

### 3. Sign the Server CSR with your Root CA
openssl x509 -req -passin pass:$password \
  -extfile <(printf "subjectAltName=DNS:localhost,IP:127.0.0.1") \
  -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt

### 4. Remove passphrase for automated service use
openssl rsa -passin pass:$password -in server.key -out server.key

## Generate client certificate

### 1. Create Client Key
openssl genrsa -passout pass:$password -des3 -out client.key 4096

### 2. Create Client CSR
openssl req -passin pass:$password -new -key client.key -out client.csr \
  -subj "/C=US/ST=CA/L=PaloAlto/O=Test/OU=Client/CN=my-client"

### 3. Sign the Client CSR with your Root CA
openssl x509 -passin pass:$password \
  -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out client.crt

### 4. Remove passphrase 
openssl rsa -passin pass:$password -in client.key -out client.key

Transmission over a non-secure channel may expose sensitive cryptographic material, including encryption keys. Configure the gRPC server and client to use TLS with certificate validation enabled before transmitting sensitive data.

3.3. Section DPL_PACKET_IO

  • enabled: When set to true, initializes the Packet IO infrastructure. This is required for P4 Controller Packet IO, the "Add Entry" feature (from DPL programs), and DPL Debugger tools.

  • dpdk_log_level: Sets the DPDK log level for Packet IO operations (e.g., *:debug). Refer to the DPDK Logging Guide for details.

3.4. Section DPL_SHM

  • enabled: When set to true, initializes the Shared Memory (SHM) infrastructure, required for controller applications utilizing the DPL Runtime Controller SDK.

  • ring_size: Controls the size of SHM requests ring/queue used for communication between a dpl-rt-controller and the dpl-rt-service for passing messages to add/delete High Update Rate (HUR) tables entries. When this ring/queue is full, entry add/delete API calls will return DOCA_ERROR_AGAIN error code, in such cases, the client application must retry calling the same operation until it gets a different error code.

3.5. Example Service Daemon Configuration File

Bash
# Example of a possible DPL RT Service GENERAL configuration file

[LOGGING]
# Path to the dpl_rtd log file.
log_file_path=/var/log/doca/dpl_rt_service/dpl_rtd.log
# Log level for the DPL RT Service.
# Possible log_level values (case insensitive):
# DISABLE
# CRITICAL
# ERROR
# WARNING
# INFO
# DEBUG
# TRACE
log_level=INFO

[P4RT_RPC_SERVER]
server_address=[::]    # IPv6 "ANY" allows IPv4 connections
server_tcp_port=9559
## If you would like to enable TLS authentication for the gRPC server connections, uncomment the following line and provided the required info:
#server_cert=/path/to/server.crt
#ca_cert=/path/to/ca.crt
#server_key=/path/to/server.key

[DPL_ADMIN_RPC_SERVER]
server_address=[::]    # IPv6 "ANY" allows IPv4 connections
server_tcp_port=9600
## If you would like to enable TLS authentication for the gRPC server connections, uncomment the following line and provided the required info:
#server_cert=/path/to/server.crt
#ca_cert=/path/to/ca.crt
#server_key=/path/to/server.key

[DPL_NSPECT_RPC_SERVER]
server_address=[::]    # IPv6 "ANY" allows IPv4 connections
server_tcp_port=9560
## If you would like to enable TLS authentication for the gRPC server connections, uncomment the following line and provided the required info:
#server_cert=/path/to/server.crt
#ca_cert=/path/to/ca.crt
#server_key=/path/to/server.key

[DPL_PACKET_IO]
# Enable Packet IO processing.
enabled=true
# DPDK --log-level value to be used when working with Packet IO.
# Uncomment and set the desired log level, for example *:debug for high Debug level logs.
# See more details and possible values at https://doc.dpdk.org/guides/prog_guide/log_lib.html
#dpdk_log_level=*:debug

# Buffer size for holding Packet-In packets to be sent to the P4Runtime Controller.
# Once this buffer is full, incoming Packet-In packets will be dropped by the P4Runtime Server.
# Note that a big buffer size can lead to a memory usage increase.
packet_in_buffer_size=131072

[DPL_SHM]
# Enable Shared Memory (SHM) infrastructure for High Update Rate (HUR) tables support using dpl-rt-controller SDK.
enabled=true
# Shared Memory ring size:
# The size of SHM requests ring used for communication between a dpl-rt-controller and the dpl-rt-service for passing
# messages to add/delete High Update Rate (HUR) tables entries.
ring_size=2048


4. System Configuration

Path: /etc/dpl_rt_service/system.conf.

This file handles hardware interaction tuning for the DPL Runtime Service.

4.1. Section LOGICAL_CORES

  • packet_io_lcores: Specifies the logical CPU core for Packet IO processing. Do not use lcore 0.

  • rule_insertion_lcores: Specifies the logical CPU core for rule insertions/deletions. Do not use lcore 0. 

    This core must be different from cores used by any controller application utilizing the DPL Runtime Controller SDK.

4.2. Section HAL

This section configures Hardware Steering (HWS) settings. 

Do not modify settings in this section unless instructed by NVIDIA Support. These settings directly impact performance characteristics such as rule update rates and latency.

  • queue_size: HWS queue size.

  • queues_num: Number of HWS queues for rule insertions/deletions.

  • burst_size: Burst size for HWS rule insertions/deletions.

4.3. Example System Configuration File 

[LOGICAL_CORES]
# Logical CPU core for processing Packet IO (do not use lcore 0).
packet_io_lcores=1
# Logical CPU core for processing rule insertions/deletions (do not use lcore 0).
rule_insertion_lcores=2

[HAL]
queue_size=1024
queues_num=1
burst_size=32

Last updated: