DOCA Switch Application Guide

This guide provides an example of switch implementation on top of NVIDIA® BlueField® DPU.

Introduction

DOCA Switch is a network application that leverages the DPU's hardware capability for internal switching between representor ports on the DPU.

DOCA Switch is based on the DOCA Flow library. As such, it exposes a command line interface which receives DOCA Flow like commands to allow adding rules in real time. 

System Design

DOCA Switch is designed to run on the DPU as a standalone application (all network traffic goes directly through it).

Traffic flows between two VMs on the host:

Traffic flow from a physical port to a VM on the host:

Application Architecture

DOCA Switch is based on 3 modules:

• Command line interface – receives pre-defined DOCA Flow-like commands and parses them

• Flow pipes manger – generates a unique identification number for each DOCA Flow structure created

• Switch core – combines all modules together and calls necessary DOCA Flow API

Port initialization cannot be made dynamically. All ports must be defined when running the application with standard DPDK flags.

• When adding a pipe or an entry, the user must run commands to create the relevant structs beforehand

• Optional parameters must be specified by the user in the command line; otherwise, NULL is used

• After a pipe or an entry is created successfully, the relevant ID is printed for future use

DOCA Libraries

This application leverages the following DOCA libraries:

• DOCA Flow

Refer to its respective programming guide for more information.

Compiling the Application

DOCA reference applications are installed with full source code and build instructions. This allows you to compile them as-is or modify the source code to create custom versions.

The source code for the application is located in the following directory: 

/opt/mellanox/doca/applications/switch/

Compiling All Applications

All DOCA applications are defined under a single meson project. So, by default, the compilation includes all of them.

To build all the applications together, run: 

cd /opt/mellanox/doca/applications/
meson /tmp/build 
ninja -C /tmp/build

Recompiling Only the Current Application

To directly build only the switch application:

cd /opt/mellanox/doca/applications/
meson /tmp/build -Denable_all_applications=false -Denable_switch=true
ninja -C /tmp/build

Alternatively, one can set the desired flags in the meson_options.txt file instead of providing them in the compilation command line:

1. Edit the following flags in /opt/mellanox/doca/applications/meson_options.txt:

   • Set enable_all_applications to false

   • Set enable_switch to true

2. Run the following compilation commands:

   cd /opt/mellanox/doca/applications/
   meson /tmp/build 
   ninja -C /tmp/build
   

   doca_switch is created under /tmp/build/switch/.

Running the Application

Prerequisites

The switch application is based on DOCA Flow. Therefore, the user is required to allocate huge pages. 

$ echo '1024' | sudo tee -a /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
$ sudo mkdir /mnt/huge
$ sudo mount -t hugetlbfs -o pagesize=2M nodev /mnt/huge

Application Execution

The switch application is provided in source form. Therefore, hence a compilation is required before the application can be executed.

1. Application usage instructions: 

   Usage: doca_switch [DPDK Flags] -- [DOCA Flags] [Program Flags]
   
   DOCA Flags:
     -h, --help                        Print a help synopsis
     -v, --version                     Print program version information
     -l, --log-level                   Set the (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
     --sdk-log-level                   Set the SDK (numeric) log level for the program <10=DISABLE, 20=CRITICAL, 30=ERROR, 40=WARNING, 50=INFO, 60=DEBUG, 70=TRACE>
     --log-filter                      Filter logs from specific modules, separated by comma
     -j, --json <path>                 Parse command line flags from an input json file
   
   Program Flags:
     -a, --device                      device
     -r, --rep                         device representor
     -exp, --expert-mode               set expert mode
   

   This usage printout can be printed to the command line using the -h (or --help) options:

   ./doca_switch -- -h
   

   
   

   For additional information, refer to section "DOCA Switch Application Guide | id (3.3.0_Jan26GA)DOCASwitchApplicationGuide CommandLineFlags".

   
   

2. CLI example for running the application on BlueField:

   ./doca_switch -- -r pci/03:00.0,pf0vf[0-1] -l 60
   

   
   

   The PCIe address  (03:00.0)  should match the address of the desired PCIe device.

   For additional information regarding the command line syntax of the device and device representor identifiers, please referr to the matching section in the DOCA Arg Parser Guide.

Command Line Flags

General Flags

Program Flags

Supported Commands

• create pipe port_id=[port_id][,<optional_parameters>]
  Available optional parameters:

  • name=<pipe-name>

  • root_enable=[1|0]

  • monitor=[1|0]

  • match_mask=[1|0]

  • fwd=[1|0]

  • fwd_miss=[1|0]

  • type=[basic|control]

• add entry pipe_id=<pipe_id>,pipe_queue=<pipe_queue>[,<optional_parameters>]
  Available optional parameters:

  • monitor=[1|0]

  • fwd=[1|0]

• add control_pipe entry priority=<priority>,pipe_id=<pipe_id>,pipe_queue=<pipe_queue>[,<optional_parameters>]
  Available optional parameters:

  • match_mask=[1|0]

  • fwd=[1|0]

• destroy pipe pipe_id=<pipe_id>

• rm entry pipe_queue=<pipe_queue>,entry_id=[entry_id]

• port pipes flush port_id=[port_id]

• port pipes dump port_id=[port_id],file=[file_name]

• query entry_id=[entry_id]

• create [struct] [field=value,…]
  Struct options: pipe_match, entry_match, match_mask, actions, monitor, fwd, fwd_miss

  • Match struct fields:

    Fields	Field Options
    flags
    port_meta
    outer.eth.src_mac
    outer.eth.dst_mac
    outer.eth.type
    outer.vlan_tci
    outer.l3_type	ipv4, ipv6
    outer.src_ip_addr
    outer.dst_ip_addr
    outer.l4_type_ext	tcp, udp, gre
    outer.tcp.flags	FIN, SYN, RST, PSH, ACK, URG, ECE, CWR
    outer.tcp_src_port
    outer.tcp_dst_port
    outer.udp_src_port
    outer.udp_dst_port
    tun_type
    vxlan_tun_id
    gre_key
    gtp_teid
    inner.eth.src_mac
    inner.eth.dst_mac
    inner.eth.type
    inner.vlan_tci
    inner.l3_type	ipv4, ipv6
    inner.src_ip_addr
    inner.dst_ip_addr
    inner.l4_type_ext	tcp, udp
    inner.tcp.flags	FIN, SYN, RST, PSH, ACK, URG, ECE, CWR
    inner.tcp_src_port
    inner.tcp_dst_port
    inner.udp_src_port
    inner.udp_dst_port

    
    

  • Actions struct fields:

    Fields	Field Options
    decap	true, false
    mod_src_mac
    mod_dst_mac
    mod_src_ip_type	ipv4, ipv6
    mod_src_ip_addr
    mod_dst_ip_type	ipv4, ipv6
    mod_dst_ip_addr
    mod_src_port
    mod_dst_port
    ttl
    has_encap	true, false
    encap_src_mac
    encap_dst_mac
    encap_src_ip_type	ipv4, ipv6
    encap_src_ip_addr
    encap_dst_ip_type	ipv4, ipv6
    encap_dst_ip_addr
    encap_tup_type	vxlan, gtpu, gre
    encap_vxlan-tun_id
    encap_gre_key
    encap_gtp_teid

    
    

  • FWD struct fields:

    Fields	Field Options
    type	rss, port, pipe, drop
    rss_flags
    rss_queues
    num_of_queues
    port_id
    next_pipe_id

    
    

  • Monitor struct fields:

    • flags

    • cir

    • cbs

    • aging

The physical port number (only one physical port is supported) will always be 0 and all representor ports are numbered from 1 to N where N is the number of representors being used. For example:

• Physical port ID: 0

• VF0 representor port ID: 1

• VF1 representor port ID: 2

• VF2 representor port ID: 3

The following is an example of creating a pipe and adding one entry into it:

create fwd type=port,port_id=0xffff
create pipe port_id=0,name=p0_to_vf1,root_enable=1,fwd=1
create fwd type=port,port_id=1
add entry pipe_queue=0,fwd=1,pipe_id=1012
          ....
rm entry pipe_queue=0,entry_id=447

1. Pipe is configured on port ID 0 (physical port).

2. Entry is configured to forward all traffic directly into port ID 1 (VF0).

3. When the forwarding rule is no longer needed, the entry is deleted.

4. Ultimately, both entries are deleted, each according to the unique random ID it was given:

Troubleshooting

Refer to the NVIDIA BlueField Platform Software Troubleshooting Guide for any issue encountered with the installation or execution of the DOCA applications.

Application Code Flow

1. Parse application argument. 

   1. Initialize the arg parser resources and register DOCA general parameters.

      doca_argp_init();
      

      
      

   2. Register DOCA Flow switch mode application parameters.

      register_doca_flow_switch_param();
      

      
      

   3. Parse app parameters.

      doca_argp_start();
      

      
      

2. Initiate DOCA Flow switch mode application parameters.
   
   

   init_doca_flow_switch_common();
   

3. Initialize DPDK ports and queues.

   dpdk_queues_and_ports_init();
   

   
   

4. Initialize DOCA Switch.

   switch_init();
   

   1. Initialize DOCA Flow.

   2. Create port pairs.

   3. Create Flow Pipes Manger module.

   4. Register an action for each relevant CLI command.

5. Initialize Flow Parser.

   flow_parser_init();
   

   1. Reset all internal Flow Parser structures.

   2. Start the command line interface.

   3. Receive user commands, parse them, and call the required DOCA Flow API command.

   4. Close the interactive shell once a "quit" command is entered.

6. Clean Flow Parser resources.

   flow_parser_cleanup();
   

   
   

7. Destroy Switch resources.

   switch_destroy();
   

   1. Destroy Flow Pipes Manager resources.

8. Destroy DOCA Flow.

   switch_destroy();
   

   
   

9. Destroy DPDK ports and queues.

   dpdk_queues_and_ports_fini();
   

   
   

10. DPDK finish.

    dpdk_fini();
    

    1. Call rte_eal_destroy() to destroy initialized EAL resources.

11. Arg parser destroy. 

    doca_argp_destroy();
    

    
    

References

• /opt/mellanox/doca/applications/switch/

Last updated: February 25, 2026