DOCA Eth L2 Forwarding Application Guide

This document provides an Ethernet L2 Forwarding implementation on top of the NVIDIA® BlueField® DPU.

Introduction

The Ethernet L2 Forwarding application is a DOCA Ethernet based application that forwards traffic from a single RX port to a single TX port and vice versa, leveraging DOCA's task/event batching feature for enhanced performance.

The application can run both on the host and the BlueField, and has two main modes:

• Two-sided forwarding – device 1 → device 2 and device 2 → device 1

• One-sided forwarding – device 1 → device 2 or device 2 → device 1

The one-sided mode offers better performance, enlarging the packets forwarding rate.

System Design

The Ethernet L2 Forwarding application runs on the host or the BlueField.

• The following diagram shows the application running on the host.
  

• The following diagram shows the application running on the BlueField:
  
  

Application Architecture

The Ethernet L2 Forwarding application runs on top of the DOCA Ethernet API to form an (two/one-sided) L2 forwarding between two ports.

1. Two DOCA devices are opened.

2. Two DOCA mmaps are created.

3. Two DOCA Flow ports are configured and started, each with a different opened DOCA device.

4. Two DOCA Ethernet TXQ and RXQ contexts are initialized, each TXQ-RXQ pair with a different opened DOCA device such that traffic is steered from the device to the corresponding RXQ, and from the corresponding TXQ to the device.

5. Forwarding - Packets received by device x are steered to RXQ x, then allocated to TXQ y and sent by device y (and vice versa).

DOCA Libraries

This application leverages the following DOCA libraries:

• DOCA Ethernet
• DOCA Flow

For additional information about the used DOCA libraries, please refer to the respective programming guides.

Compiling the Application

DOCA reference applications are installed with full source code and build instructions. This allows you to compile them as-is or modify the source code to create custom versions.

The source code for the application is located in the following directory: 

/opt/mellanox/doca/applications/eth_l2_fwd/

Compiling All Applications

The applications are all defined under a single meson project, meaning that the default compilation will compile all the DOCA applications.

To build all the applications together, run: 

cd /opt/mellanox/doca/applications/
meson /tmp/build 
ninja -C /tmp/build

Compiling Only the Current Application

1. To directly build only the Ethernet L2 Forwarding application:

   cd /opt/mellanox/doca/applications/
   meson /tmp/build -Denable_all_applications=false -Denable_eth_l2_fwd=true
   ninja -C /tmp/build
   

   doca_eth_l2_fwd will be created under /tmp/build/eth_l2_fwd/.

2. Alternatively, one can set the desired flags in the meson_options.txt file instead of providing them in the compilation command line:Edit the following flags in /opt/mellanox/doca/applications/meson_options.txt:Set enable_all_applications to falseSet enable_eth_l2_fwd to trueThe same compilation commands should be used, as were shown in the previous section: cd /opt/mellanox/doca/applications/ meson /tmp/build ninja -C /tmp/build doca_eth_l2_fwd will be created under /tmp/build/eth_l2_fwd/.

Running the Application

Application Execution

The Ethernet L2 Forwarding application is provided in source form, hence a compilation is required before the application can be executed.

• Application usage instructions: 

  Usage: doca_eth_l2_fwd [DOCA Flags] [Program Flags]
  
  DOCA Flags:
    -h, --help                        Print a help synopsis
    -v, --version                     Print program version information
    -l, --log-level                   Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
    --sdk-log-level                   Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
    --log-filter                      Filter logs from specific modules, separated by comma
    -j, --json <path>                 Parse command line flags from an input json file
  
  Program Flags:
    -d, --devs-names <name1,name2>    Set two IB devices names separated by a comma, without spaces.
    -r, --rate <rate>                 Set packets receive rate (in [MB/s]), default is 12500.
    -ps, --pkt-size <size>            Set max packet size (in [B]), default is 1600.
    -t, --time <time>                 Set packet max process time (in [μs]), default is 1.
    -nt, --num-tasks <num>            Set number of tasks per batch, default is 128.
    -nb, --num-batches <num>          Set number of task batches, default is 32.
    -o, --one-sided-forwarding <num>  Set one-sided forwarding: 0 - two-sided forwarding, 1 - device 1 -> device 2, 2 - device 2 -> device 1. default is 0.
    -f, --max-forwardings <num>       Set max forwardings after which the application run will end, default is 0, meaning no limit.
  

  For additional information, please refer to the DOCA Eth L2 Forwarding Application Guide | Command Line Flags section below.

  The above usage printout can be printed to the command line using the -h (or --help) options:

  ./doca_eth_l2_fwd -h
  

  
  

  
  

• CLI example for running the application either on the BlueField or on the host:

  ./doca_eth_l2_fwd -d mlx5_0,mlx5_1
  

  Both IB devices identifiers (mlx5_0, mlx5_1) should match the identifiers of the desired IB devices.

Command Line Flags

General Flags

Program Flags

Troubleshooting

Refer to the NVIDIA BlueField Platform Software Troubleshooting Guide for any issue encountered with the compilation, installation, or execution of the DOCA applications.

Application Code Flow

1. Parse application argument. 

   1. Initialize Arg parser resources and register DOCA general parameters.

      doca_argp_init();
      

      
      

   2. Register Ethernet L2 Forwarding application parameters.

      register_eth_l2_forwarding_params();
      

      
      

   3. Parse the arguments.

      doca_argp_start();
      

      1. Parse DOCA flags.

      2. Parse application parameters.

2. Execute Ethernet L2 Forwarding application main logic.

   eth_l2_fwd_execute();
   

   1. Open the two chosen DOCA devices.

   2. Initialize necessary DOCA Core objects.

   3. Initialize ETH RXQ/TXQ contexts for the devices.

   4. Forward packets.

3. Clean up application resources.

   eth_l2_fwd_cleanup();
   

   1. Stop all contexts and drain tasks.

   2. Free all application resources.

4. Arg parser destroy.

   doca_argp_destroy()
   

References

• /opt/mellanox/doca/applications/eth_l2_fwd/

Last updated: May 27, 2026