Networking DOCA SDK Documentation
Networking DOCA SDK Documentation
Breadcrumbs

Adaptive Retransmission: Parameters Control

Introduction

Adaptive Retransmission (ADP) Algorithm timeout adjustments is introduced in the April 2025 GA firmware for all NICs supporting the standard ADP algorithm. It addresses advanced customer needs for greater flexibility, enabling multiple timeout values and configurable retry counts per timeout. This enhancement allows for faster reactions to dropped packet reports while giving the transport layer the ability to defer flow errors. It assumes that QP users understand their fabric's characteristics and can select appropriate parameter values to optimize performance.

ROCE_ACCL Layout and Relevant Field Description

ROCE_ACCL Register

RoCE-ACCL-Register-Layout.png

Offset

Bits

Name

Description

Access

00h

28

adp_retx_profile_select

In query, indicates that adp_retx_profile_id and adp_retx_profile are supported and valid.

In set, indicates that the profile indicated adp_retx_profile_id is modified with the parameters in adp_retx_profile.

RW

0

roce_adp_retrans_field_select

In Query, indicates if roce_adp_retrans mode is configurable.

In set, indicates roce_adp_retrans_en field is modified.

RW

04h

30:28

adp_retx_profile_id

The index of the adaptive-retransmission profile to query or to modify. Value 0 is reserved.

Valid if adp_retrans_timeouts_profile_select is set. Must comply with adp_retx_profile_max_id.

Index

0

roce_adp_retrans_en

If set, the adaptive retransmission window functionality is enabled.

Valid only when roce_adp_retrans_field_select == 1.

RW

08h

30:28

adp_retx_profile_max_range_num

Maximal number of timeout ranges in adp_retx_profile.

RO

26:24

adp_retx_profile_max_id

Maximal index supported for adp_retx_profile_id.

RO

19:0

adp_retx_base_timeout_min

The minimal base-timeout value for adp_retx_profile, given in nano second units.

RO

10h-3Ch

384

adp_retx_profile

Adaptive Retransmission: Parameters Control | Adaptive Retransmission Profile.

RW

Adaptive Retransmission Profile

Adaptive-Retransmission-Profile-Layout.png

Offset

Bits

Name

Description

Access

00h

31

qp_total_timeout

If set, Total Retransmission Timeout will use QP parameters as timeout value, given by:

  • QP.retry_count * QP.timeout or
    QP.extended_retry_count * QP.timeout

  • If clr, retx_total_timeout is used


30:28

range_num

Number of valid ranges in timeout_range[] array.


26:24

start_range_index

Index of timeout_range used as the first range following QP creation.


23:22

time_unit

Time granularity of time_base

  • 0x1: TIME_USEC – μsecs

  • Other values are reserved


15:0

time_base

Base value for all time parameter, units defined by time_unit.


04h

31:24

retx_total_timeout

Total Retransmission Timeout, given by base_time << retx_total_timeout.

Valid if qp_total_timeout is 0x0.


15:8

timeout_init_low_bound

Lower bound for QP initial timeout randomization, given by base_time << timeout_init_low_bound


7:0

timeout_init_range_size

Upper bound for QP initial timeout randomization, given by
base_time << (timeout_init_low_bound + timeout_init_range_size)


08h-14h

31:0

timeout_range[4]

Array of timeout ranges. First range_num entries are valid.


Adaptive Retransmission Timeout Range

Offset

Bits

Name

Description

Access

00h

30:28

prev_range_index

Index of the timeout range to used when the current timeout is the range's lower-bound, and timeout should be decreased.


27:26

dec_mode

Defines the behavior of timeout decrease until lower bound is reached.

  • 0x0: TO_DIV_4

  • 0x1: TO_DIV_2

  • 0x2: TO_LOW_BOUND

  • Other values are reserved


25:16

timeout_retry_num

Number of consecutive retransmissions attempts with the same timeout value, before timeout is increased.


15:8

range_low_bound

Lower bound of the timeout range, given bybase_time << range_low_bound


7:0

range_size

Upper bound of the timeout range given by base_time << (range_low_bound + range_size)


Enablement and Capabilities

The following are ADP Retransmission configuration details:

  • roce_adp_retrans_field_select – Indicates that ADP Retransmission functionality is supported by the device.

  • adp_retx_profile_select – Indicates support for advanced ADP Retransmission profile configuration.

  • roce_adp_retrans_en – Enables ADP Retransmission.

  • adp_retx_profile_id – Activates an advanced ADP Retransmission profile with the specified ID. Currently, only profile #1 is available for users; profile #0 is reserved as the default.

  • adp_retx_base_timeout_min – Minimum base timeout enforced by firmware: 0xFA0 (4000 nanoseconds, i.e., 4 microseconds).

  • adp_retx_profile_max_id – Maximum of 1 profile supported by the firmware.

  • adp_retx_profile_max_range_num – Maximum of 4 timeout ranges supported per profile.

Algorithm & Profile Description and Formulas

The following is the adaptive retransmission (ADP) timeout algorithm flow.

Initialization

  • Initial timeout value:

    rand(profile.timeout_init_low_bound, profile.timeout_init_low_bound + profile.timeout_init_range_size - 1)

  • Hardware timer: 

    profile.time_base * (2 ^ granularity)

    Granularity = initial timeout.

  • On the first timeout event:

    • If current_timeout falls within any defined range (range_low_bound → range_low_bound + range_size), the algorithm continues within that range.

    • Otherwise, it starts from profile.timeout_range[profile.start_range_index].range_low_bound.

Notes:

  • Initial timeout is randomized to prevent multiple QPs from starting with the same value.

  • A good configuration ensures the initial timeout falls within a defined range.

  • The initial timeout value is retried only once.

  • time_base must be a power of 2, with the minimum allowed value per firmware capability (4 µs).

On Consecutive Timeout Events

  • current_timeout is used profile.timeout_range[current_range].timeout_retry_num times.

  • Once exhausted, current_timeout is doubled until it reaches

    profile.timeout_range[current_range].range_low_bound + profile.timeout_range[current_range].range_size

  • The algorithm then continues with the next defined range.

Notes:

  • Each range defines one or more timeout values.

  • Timeout ranges must be sorted by range_low_bound (range_low_bound[i] < range_low_bound[j] for i < j).

  • Maximum timeout is capped by QP.ack_timeout, regardless of configured ranges.

  • The overall timeout is determined by:

    • profile.qp_total_timeout ? (QP.ack_timeout * QP.retry_num) : profile.retx_total_timeout

  • If no forward progress is made within the total timeout, the QP fails and transitions to ERR state with error code IBV_WC_RETRY_EXC_ERR (Transport Retry Counter Exceeded).

On Forward Progress (e.g., ACK Received)

  • current_timeout is decremented based on profile.timeout_range[current_range].dec_mode:

    • Decrement by 2

    • Decrement by 4

    • Or reset to profile.timeout_range[current_range].range_low_bound of the current range.

  • If current_timeout reaches the lowest timeout in the range, move to the previous range defined by profile.timeout_range[current_range].prev_range_index.

Notes:

  • The minimum enforced timeout is adp_retx_base_timeout_min (4 µs).

  • prev_range_index must always be less than current_range.

General Notes

  • If no profile is selected, the ADP algorithm defaults to firmware-defined timeouts.

  • When a profile is selected, it applies to all QPs of the PF and its associated VFs.

  • Reverting to firmware-defined timeouts requires a fwreset or system reboot.

  • To check which mode is active:

    • adp_retx_profile_id = 0 → firmware-defined timeouts

    • adp_retx_profile_id > 0 → profile-based timeouts

Sample Usage

The following examples show how to review and update the Adaptive Retransmission (ADP) profile using both:

  • The standard ROCE_ACCL register

  • The advanced mechanism (reg_id 0x402c)

The examples below use mlxreg, but the same usage applies when using the mstreg tool.

[]$ sudo mlxreg -d 1a:00.0 --yes --reg_name ROCE_ACCL --get
Sending access register...
 
Field Name                                 	| Data	
============================================================
roce_adp_retrans_field_select              	| 0x00000001
roce_tx_window_field_select                	| 0x00000001
roce_slow_restart_field_select             	| 0x00000001
roce_slow_restart_idle_field_select        	| 0x00000001
min_ack_timeout_limit_disabled_field_select	| 0x00000001
adaptive_routing_forced_en_field_select    	| 0x00000000
selective_repeat_forced_en_field_select    	| 0x00000001
dc_half_handshake_en_field_select          	| 0x00000000
cc_per_plane_en_field_select               	| 0x00000000
ack_dscp_force_field_select                	| 0x00000001
cc_burst_size_select                       	| 0x00000001
ps_metadata_reg_id_select                  	| 0x00000000
ifa2_turn_around_select                    	| 0x00000001
adp_retx_profile_select                    	| 0x00000001
roce_adp_retrans_en                        	| 0x00000001
roce_tx_window_en                          	| 0x00000000
roce_slow_restart_en                       	| 0x00000001
roce_slow_restart_idle_en                  	| 0x00000000
min_ack_timeout_limit_disabled             	| 0x00000000
adaptive_routing_forced_en                 	| 0x00000000
selective_repeat_forced_en                 	| 0x00000000
dc_half_handshake_en                       	| 0x00000000
cc_per_plane_en                            	| 0x00000000
ack_dscp_force                             	| 0x00000000
ack_dscp                                   	| 0x00000000
ps_metadata_reg_id                         	| 0x00000000
cc_burst_size                              	| 0x00000000
Ifa2_turn_around_en                        	| 0x00000000
adp_retx_profile_id                        	| 0x00000000
adp_retx_base_timeout_min                  	| 0x00000fa0
adp_retx_profile_max_id                    	| 0x00000001
adp_retx_profile_max_range_num             	| 0x00000004
time_base                                  	| 0x00000000
time_unit                                  	| 0x00000000
start_range_index                          	| 0x00000000
range_num                                  	| 0x00000000
qp_total_timeout                           	| 0x00000000
timeout_init_range_size                    	| 0x00000000
timeout_init_low_bound                     	| 0x00000000
retx_total_timeout                         	| 0x00000000
range_size_0                               	| 0x00000000
range_low_bound_0                          	| 0x00000000
timeout_retry_num_0                        	| 0x00000000
dec_mode_0                                 	| 0x00000000
prev_range_index_0                         	| 0x00000000
range_size_1                               	| 0x00000000
range_low_bound_1                          	| 0x00000000
timeout_retry_num_1                        	| 0x00000000
dec_mode_1                                 	| 0x00000000
prev_range_index_1                         	| 0x00000000
range_size_2                               	| 0x00000000
range_low_bound_2                          	| 0x00000000
timeout_retry_num_2                        	| 0x00000000
dec_mode_2                                 	| 0x00000000
prev_range_index_2                         	| 0x00000000
range_size_3                               	| 0x00000000
range_low_bound_3                          	| 0x00000000
timeout_retry_num_3                        	| 0x00000000
dec_mode_3                                 	| 0x00000000
prev_range_index_3                         	| 0x00000000
============================================================
 
[]$ sudo mlxreg -d 1a:00.0 --yes --reg_id 0x402c --reg_len 0x40 --get
Sending access register...
 
Address	| Data
=======================
0x00000000 | 0x1800065f
0x00000004 | 0x00000005
0x00000008 | 0x41000fa0
0x0000000c | 0x00000000
0x00000010 | 0x00000000
0x00000014 | 0x00000000
0x00000018 | 0x00000000
0x0000001c | 0x00000000
0x00000020 | 0x00000000
0x00000024 | 0x00000000
0x00000028 | 0x00000000
0x0000002c | 0x00000000
0x00000030 | 0x00000000
0x00000034 | 0x00000000
0x00000038 | 0x00000000
0x0000003c | 0x00000000
=======================
 
// Equivalent usage with ROCE_ACCL reg_name commented out for clarity and future usage
[]$ ###sudo mlxreg -d 1a:00.0 --reg_name ROCE_ACCL --set roce_adp_retrans_en=0x1,adp_retx_profile_id=0x1,time_base=4,time_unit=1,start_range_index=0,range_num=1,qp_total_timeout=1,timeout_init_range_size=2,timeout_init_low_bound=0,range_size_0=0x2,range_low_bound_0=0,timeout_retry_num_0=0x1,dec_mode_0=1,prev_range_index_0=0
 
[]$ sudo mlxreg -d 1a:00.0 --reg_id 0x402c --reg_len 0x40 --set "0x4.0:1=1,0x4.28:3=1,0x10.0:16=4,0x10.22:2=1,0x10.24:3=0,0x10.28:3=1,0x10.31:1=1,0x14.0:8=2,0x14.8:8=0,0x18.0:8=2,0x18.8:8=0,0x18.16:10=1,0x18.26:2=1"
You are about to send access register id: 0x402c with the following data:
Address	| Data
=======================
0x00000000 | 0x1800065f
0x00000004 | 0x10000005
0x00000008 | 0x41000fa0
0x0000000c | 0x00000000
0x00000010 | 0x90400004
0x00000014 | 0x00000002
0x00000018 | 0x04010002
0x0000001c | 0x00000000
0x00000020 | 0x00000000
0x00000024 | 0x00000000
0x00000028 | 0x00000000
0x0000002c | 0x00000000
0x00000030 | 0x00000000
0x00000034 | 0x00000000
0x00000038 | 0x00000000
0x0000003c | 0x00000000
=======================
 
 Do you want to continue ? (y/n) [n] : y
 Sending access register...
[]$ sudo mlxreg -d 1a:00.0 --yes --reg_name ROCE_ACCL --get
Sending access register...
 
Field Name                                 	| Data	
============================================================
roce_adp_retrans_field_select              	| 0x00000001
roce_tx_window_field_select                	| 0x00000001
roce_slow_restart_field_select             	| 0x00000001
roce_slow_restart_idle_field_select        	| 0x00000001
min_ack_timeout_limit_disabled_field_select	| 0x00000001
adaptive_routing_forced_en_field_select    	| 0x00000000
selective_repeat_forced_en_field_select    	| 0x00000001
dc_half_handshake_en_field_select          	| 0x00000000
cc_per_plane_en_field_select               	| 0x00000000
ack_dscp_force_field_select                	| 0x00000001
cc_burst_size_select                       	| 0x00000001
ps_metadata_reg_id_select                  	| 0x00000000
ifa2_turn_around_select                    	| 0x00000001
adp_retx_profile_select                    	| 0x00000001
roce_adp_retrans_en                        	| 0x00000001
roce_tx_window_en                          	| 0x00000000
roce_slow_restart_en                       	| 0x00000001
roce_slow_restart_idle_en                  	| 0x00000000
min_ack_timeout_limit_disabled             	| 0x00000000
adaptive_routing_forced_en                 	| 0x00000000
selective_repeat_forced_en                 	| 0x00000000
dc_half_handshake_en                       	| 0x00000000
cc_per_plane_en                            	| 0x00000000
ack_dscp_force                             	| 0x00000000
ack_dscp                                   	| 0x00000000
ps_metadata_reg_id                         	| 0x00000000
cc_burst_size                              	| 0x00000000
Ifa2_turn_around_en                        	| 0x00000000
adp_retx_profile_id                        	| 0x00000001
adp_retx_base_timeout_min                  	| 0x00000fa0
adp_retx_profile_max_id                    	| 0x00000001
adp_retx_profile_max_range_num             	| 0x00000004
time_base                                  	| 0x00000004
time_unit                                  	| 0x00000001
start_range_index                          	| 0x00000000
range_num                                  	| 0x00000001
qp_total_timeout                           	| 0x00000001
timeout_init_range_size                    	| 0x00000002
timeout_init_low_bound                     	| 0x00000000
retx_total_timeout                         	| 0x00000000
range_size_0                               	| 0x00000002
range_low_bound_0                          	| 0x00000000
timeout_retry_num_0                        	| 0x00000001
dec_mode_0                                 	| 0x00000001
prev_range_index_0                         	| 0x00000000
range_size_1                               	| 0x00000000
range_low_bound_1                          	| 0x00000000
timeout_retry_num_1                        	| 0x00000000
dec_mode_1                                 	| 0x00000000
prev_range_index_1                         	| 0x00000000
range_size_2                               	| 0x00000000
range_low_bound_2                          	| 0x00000000
timeout_retry_num_2                        	| 0x00000000
dec_mode_2                                 	| 0x00000000
prev_range_index_2                         	| 0x00000000
range_size_3                               	| 0x00000000
range_low_bound_3                          	| 0x00000000
timeout_retry_num_3                        	| 0x00000000
dec_mode_3                                 	| 0x00000000
prev_range_index_3                         	| 0x00000000
============================================================

Last updated: