Networking DOCA SDK Documentation

DOCA DevEmu NVMe

Introduction

The NVIDIA DOCA™ DevEmu NVMe library provides low-level C APIs for implementing an emulated NVMe controller on NVIDIA® BlueField®. It manages the host-facing controller interface and accelerated command/completion queue processing. Applications receive NVMe requests, execute their command semantics and return completions.

The library is command-set agnostic. It delivers Admin and I/O submission queue entries to the application rather than implementing a storage backend or a namespace service. The application determines which command sets and commands its controller supports.

Library Interaction

  • DOCA DevEmu PCI supplies PCI function discovery, representors, host visibility, MSI-X configuration and function-level reset events.

  • DOCA DevEmu NVMe adds controller-register access and notifications, NVMe queue objects, request delivery and completion posting.

  • DOCA Core supplies contexts, progress engines, buffers, memory maps and asynchronous execution. The application uses suitable data-movement APIs, such as DOCA DMA, to access command payloads.

Application Responsibilities

  • Implement controller properties and the NVMe controller enable/disable and shutdown handshakes.

  • Implement Admin commands, including Identify, feature negotiation and creation/deletion of I/O queues; advertise only capabilities the application actually supports.

  • Decode command opcodes, namespace identifiers and PRP/SGL data pointers; validate commands before accessing host memory.

  • Execute commands against the application's backend, finish payload transfers, and complete each delivered request exactly once.

  • Coordinate queue lifetimes, reset handling and application-owned resources on their owning threads.

The interfaces described here are marked DOCA_EXPERIMENTAL. Use the public headers matching the installed SDK. Library quality levels are listed in DOCA Libraries.

Prerequisites

Before proceeding, read DOCA Core, DOCA Device Emulation and DOCA DevEmu PCI. Understand DOCA context state transitions, progress engines, device discovery, representor lifetimes and host-memory access.

  • Familiarity with the NVMe controller-register interface, Admin commands, submission/completion queues and the command sets implemented by the application.

  • A supported BlueField emulation manager and an NVMe-capable host driver.

  • Application-owned progress loops and data-movement resources that remain valid while requests are outstanding.

Environment

Run on BlueField in DPU mode, with the privileges required for device emulation. See BlueField Modes of Operation and DOCA Compatibility Policy. This guide uses the built-in native NVMe PCI type.

Firmware Configuration

Configure native NVMe emulation before discovering or creating functions. Run MST and identify the device corresponding to the BlueField being configured; replace the placeholder below with that device's MST path.

Bash
mst start
mst status -v
mlxconfig -d /dev/mst/<device> -e query

Choose one of the following provisioning paths. Static PF provisioning and dynamic hotplug are different paths; do not combine their PF-count settings.

For command sets other than standard NVM, configure NVME_EMULATION_IOCSS as required by the desired command set. No explicit setting is required for standard NVM operation.

Static PF Configuration

The following example provisions one firmware-created NVMe PF with 32 MSI-X vectors. Choose resource counts to fit the application's queue and interrupt requirements.

A static NVMe PF is visible to the host from boot and remains present even when the emulation application is not running. If no application is available to service the controller, host firmware or the operating system may wait for it to become ready, delaying or stalling the host boot process.

Bash
mlxconfig -d /dev/mst/<device> --yes set \
    NVME_EMULATION_ENABLE=1 \
    NVME_EMULATION_NUM_PF=1 \
    NVME_EMULATION_NUM_MSIX=32

Dynamic Hotplug Configuration

The application creates the function and explicitly hotplugs it when ready to serve host requests. One switch port is reserved; the number of available hotplug functions is PCI_SWITCH_EMULATION_NUM_PORT - 1.

Bash
mlxconfig -d /dev/mst/<device> --yes set \
    NVME_EMULATION_ENABLE=1 \
    NVME_EMULATION_NUM_PF=0 \
    PCI_SWITCH_EMULATION_ENABLE=1 \
    PCI_SWITCH_EMULATION_NUM_PORT=2

Power-cycle the host server containing BlueField after these firmware changes. A DPU reboot alone is insufficient. Query the configuration again and verify that the requested values appear in Current, not only in Next Boot.

Host PCIe hotplug prerequisites, including applicable IOMMU and PCI resource-allocation settings, are described in DOCA DevEmu PCI.

Architecture

Objects

Object

Purpose

Lifecycle

doca_devemu_pci_type

Built-in native NVMe function type associated with an emulation manager.

Acquired already started; shared and library-owned.

doca_devemu_nvme_dev

One emulated NVMe controller associated with a representor.

A DOCA context, with a separate operational enable/disable state.

doca_devemu_nvme_io

Request delivery and completion processing for bound queues.

An independently created DOCA context owned by one application thread.

doca_devemu_nvme_cq

Completion queue associated with one NVMe device.

Created disabled; bind to an I/O context before enabling.

doca_devemu_nvme_sq

Submission queue permanently associated with a CQ.

Created disabled; binds to the I/O context already owning that CQ.

doca_devemu_nvme_req

One delivered host command.

Application-owned from request notification until completion.

Queue Lifecycle

  • Create: establish the queue configuration.

  • Bind: associate the queue with its I/O owner.

  • Enable: permit queue processing.

Object Relationships

  • Queues are not separate DOCA contexts.

  • I/O contexts are created independently, not as children of an NVMe device.

  • Binding a device's CQ to an I/O context establishes their association. An SQ's I/O-context association is derived from its CQ; the SQ cannot select a different I/O context.

Threading and Progress

The NVMe library uses application-driven progress and thread-owned I/O contexts. The following rules govern access to library objects and completion of asynchronous operations:

  • Serialize device-management calls with progression of the device's control-path progress engine.

  • Each I/O context belongs to one application thread. Perform queue operations and request completion on the owning thread, as required by the corresponding API.

  • An SQ and its associated CQ use the same I/O context.

  • Continue progressing the device and affected I/O contexts during request processing, asynchronous disable and teardown.

  • Do not call doca_pe_progress() recursively from a callback.

  • If backend work executes on another thread, dispatch request completion back to the request's owning I/O thread.

For parallel processing, an application may distribute queue pairs across independently owned I/O contexts. This is an optional execution arrangement, not a library requirement.

Initialization

Initialize the library and independently owned I/O contexts before discovering the emulated function and creating its NVMe device.

Library Configuration and Initialization

Discover DOCA devices and check doca_devemu_nvme_cap_is_default_nvme_pci_type_supported(). Check both the return code and the returned support flag. Open the selected emulation manager with doca_dev_open() and register it with doca_devemu_nvme_add_dev().

Apply global configuration before doca_devemu_nvme_init(). Configuration is frozen while the library is initialized; check initialization errors before creating dependent resources.

Configuration

API / behavior

Emulation managers

doca_devemu_nvme_add_dev() registers each manager used by the library.

Per-request application storage

doca_devemu_nvme_set_req_user_data_size() optionally reserves application scratch space alongside each request in the same allocation. The request handler receives a pointer to this space, valid until request completion. The default size is zero, yielding a NULL pointer. Initialize the contents when needed; the buffer is not cleared between reuse.

Maximum I/O queue size

doca_devemu_nvme_set_max_io_queue_size() uses zero-based MQES units. Check doca_devemu_nvme_cap_get_max_io_queue_size().

Entry sizes

Use the supported 64-byte SQE and 16-byte CQE sizes. Capability queries report log2 sizes 6 and 4 respectively.

After applying the global settings, call doca_devemu_nvme_init() and check the return value before creating dependent resources.

I/O Context Initialization

  1. Call doca_devemu_nvme_io_create(). The API takes no device or queue argument.

  2. While Idle, register doca_devemu_nvme_io_event_nvme_req_register().

  3. Connect doca_devemu_nvme_io_as_ctx() to the owning thread's progress engine using doca_pe_connect_ctx(), then start the context.

  4. Bind queues only after the I/O context is started. No required-I/O-context count or device-start barrier is configured through this NVMe I/O API.

Device and Function Discovery

  1. Acquire the started built-in type with doca_devemu_nvme_find_default_nvme_pci_type_by_dev(). Do not stop, reconfigure or destroy this shared library-owned type as though it were a custom type.

  2. Enumerate representors for that type using doca_devemu_pci_type_rep_list_create(), select the intended function by its identity and open it using doca_dev_rep_open(). Use the corresponding list-destruction API after selection.

For a dynamic function, first check doca_devemu_nvme_cap_is_default_nvme_pci_type_hotplug_supported(). Create the representor with the supported PCI type representor-creation API, then create and configure the NVMe device before making it visible to the host. Use the hotplug state-change event to track asynchronous visibility changes. A firmware-provisioned static PF is discovered rather than created and hotplugged.

NVMe Device Initialization

  1. Create a control-path progress engine, then call doca_devemu_nvme_dev_create(pci_type, dev_rep, pe, &nvme_dev). The device creation API receives the progress engine directly.

  2. While the device is Idle, call doca_devemu_nvme_dev_set_num_queue_pairs() with a nonzero count including the Admin queue pair. For example, one Admin pair plus four I/O pairs requires a count of five. Stay within doca_devemu_nvme_cap_get_default_nvme_pci_type_max_num_queue_pairs().

  3. Register host register-write notifications with doca_devemu_nvme_dev_event_reg_change_register(). Register device/SQ/CQ operational state-change handlers if using asynchronous disable notifications.

  4. Obtain the PCI device with doca_devemu_nvme_dev_as_pci_dev(). Register the PCI FLR handler using doca_devemu_pci_dev_event_flr_register() and, for a dynamic function, the hotplug state-change handler.

  5. Apply supported PCI endpoint configuration before host visibility locks those properties. Size MSI-X resources and queue limits consistently.

  6. Call doca_ctx_start() on doca_devemu_nvme_dev_as_ctx() and progress until Running. A Running context initially has NVMe operational state Disabled; it is not yet a ready host controller.

Queue Creation, Binding and Enabling

Initialize the controller properties implemented by the application. For a dynamic function, make it visible to the host only after controller event handling is ready.

Handle the host's controller-enable sequence, construct the Admin queue pair from the host's registers and enable processing. Receive Admin commands and create additional I/O queue pairs on demand. Create each CQ before its SQ, bind the CQ to the intended I/O context and bind the SQ to that same owner through its CQ. Enable the CQ before its SQ.

See Controller Register Changes and Creating and Binding Queues for the execution-time flows and API details.

Execution Phase

The application advances NVMe operation in response to controller-register changes, request arrivals and lifecycle events. Each flow below identifies its trigger, callback registration, application actions and completion condition.

Register device event handlers while the device context is Idle. Perform lifecycle operations on the appropriate owner thread, defer cleanup outside callbacks where necessary, and do not recursively call doca_pe_progress() from a callback.

Controller Register Changes

Trigger and registration: a host controller-register write is reported through the callback registered with doca_devemu_nvme_dev_event_reg_change_register().

Host register-write notifications supply the register offset. Read the current register through doca_devemu_nvme_dev_read_reg32() or doca_devemu_nvme_dev_read_reg64(); use the matching write API for application-controlled properties.

Register group

Application handling

CAP / VS

Report supported controller capabilities and version consistently with the implemented controller.

CC / CSTS

Implement controller enable, disable and shutdown state; do not report Ready before the Admin path can process commands.

AQA / ASQ / ACQ

Validate Admin queue sizes and addresses and construct the Admin queue pair.

CC.IOSQES / CC.IOCQES

Validate the requested entry sizes against 64-byte SQEs and 16-byte CQEs. Per-device entry-size setters are not supported in this release.

Controller Enable

When the host enables the controller, validate its configuration, create and bind the Admin CQ and SQ, then call doca_devemu_nvme_dev_enable(). It enables bound CQs before their SQs; unbound queues remain disabled. Report CSTS.RDY only after the application's enable sequence succeeds.

Completion condition: report CSTS.RDY only when the controller can process Admin commands. A Running DOCA context alone does not establish controller readiness.

Controller Disable

Trigger: the host clears CC.EN. This requests controller disable, not PCI FLR.

Follow Disabling and Draining Processing, then release controller queue resources as required by the NVMe controller-disable handshake using Removing a Queue Pair. Update CSTS.RDY after the disable sequence completes. Do not treat controller disable as a request to tear down the entire application or its independently owned I/O contexts.

Shutdown Notification

Trigger: a change to the shutdown-notification field in CC. Handle the requested shutdown according to the NVMe protocol and the application's backend requirements. Use Disabling and Draining Processing where quiescing is required, and report the corresponding shutdown status in CSTS only after the required work completes. Shutdown notification is distinct from PCI FLR.

NVMe Request Arrival

Trigger and registration: progressing an I/O context delivers a host command through the handler registered with doca_devemu_nvme_io_event_nvme_req_register(). Use the SQ's application state to route Admin and I/O requests to the appropriate handler.

Admin Requests and I/O Queue Management

Implement the Admin commands advertised by the controller. For Create I/O Completion Queue and Create I/O Submission Queue commands, validate the host parameters and construct the requested resources as described below. Complete each Admin request with the result of the operation; do not report success if queue construction failed.

For Delete I/O Submission Queue and Delete I/O Completion Queue commands, follow Removing a Queue Pair. Preserve the SQ-before-CQ dependency order, and complete a deletion command only after the requested queue has been safely removed.

Creating and Binding Queues

Translate host queue-creation parameters into the SDK's creation-attribute objects. CQ creation precedes SQ creation, and CQ binding precedes SQ binding.

  1. Allocate doca_devemu_nvme_cq_create_attr; set QID, queue size, interrupt vector/enable, physical contiguity and PRP1. Call doca_devemu_nvme_cq_create_from_dev() with ordinary creation flags for a new host queue. Free the temporary attributes after the call returns.

  2. Bind the disabled CQ with doca_devemu_nvme_cq_bind(cq, io) on the I/O owner thread.

  3. Allocate doca_devemu_nvme_sq_create_attr; set the associated CQ, QID, queue size, priority, physical contiguity and PRP1. Call doca_devemu_nvme_sq_create(), then free the temporary attributes.

  4. Call doca_devemu_nvme_sq_bind(sq, sq_user_data). The SQ inherits its CQ's I/O context; there is no separate I/O-context argument.

  5. For a newly created pair on an already enabled device, enable the CQ with doca_devemu_nvme_cq_enable() before enabling the SQ with doca_devemu_nvme_sq_enable(). Check every return value before completing the host's queue-creation command.

Queue-size units require care: the queue-creation attribute setters take an entry count in this implementation, whereas NVMe command QSIZE fields and AQA sizes are zero-based. Convert and range-check the host value before passing it to the SDK. The global maximum-I/O-queue setting remains zero-based.

Use physically contiguous queue memory, one SQ per CQ and equal SQ/CQ entry counts for this release. Validate host queue requests before creating resources; never acknowledge a queue the application failed to construct. See Known Issues for the affected queue configurations.

Request Processing

  1. Progress the owning I/O engine. The registered handler receives req, the raw SQE in host little-endian byte order and the configured per-request user-data buffer.

  2. Decode and validate the command. The SQE remains valid throughout the request's lifetime, until request completion. Obtain SQ-specific application state with doca_devemu_nvme_req_get_sq_user_data().

  3. Decode and validate the command's data pointers, then perform the required payload transfers and backend work using application-managed data-movement resources.

  4. After the operation and all required payload transfers complete, call doca_devemu_nvme_req_complete(req, dw0, dw1, status) on the request's I/O owner thread.

  5. Continue progressing the I/O engine so completion data and interrupts reach the host. Returning from the completion API does not mean the host has consumed the CQE.

Ownership of the request and its per-request application storage passes to the application on notification and returns to the library on completion. Do not retain or access those objects afterward.

The completion API takes dw0 and dw1 in little-endian byte order. It takes the 15-bit NVMe status in native CPU byte order, without the phase tag; zero means success. The library supplies the phase tag and constructs the CQE. Decode and encode status deliberately rather than passing an unconverted host-format DW3.

doca_devemu_nvme_req_get_id() is a library-assigned identifier scoped to the SQ, not the host's reusable Command Identifier (CID). Read it while the application owns the request. Applications coordinating takeover may use it for replay tracking; it does not itself implement backend deduplication.

Completion condition: complete each delivered request exactly once, after its command semantics and required payload transfers have finished. Do not access the request, its SQE or per-request user-data buffer after completion.

Asynchronous Disable Completion

Trigger: an asynchronous device or queue disable operation finishes. Register the following handlers while the NVMe device is Idle:

  • Device: doca_devemu_nvme_dev_event_state_change_register().

  • SQ: doca_devemu_nvme_dev_event_sq_state_change_register().

  • CQ: doca_devemu_nvme_dev_event_cq_state_change_register().

Application handling: query the affected object's operational state. Once it is Disabled, advance the pending controller-disable, queue-removal or teardown operation. A notification does not destroy queues or release application-owned resources. Follow Queue Draining and Teardown for the shared dependency order; do not unbind or destroy a CQ while its SQs still depend on it.

Completion condition: the affected object has reached Disabled and the application may perform the next permitted lifecycle operation. Check every disable call's return value: synchronous success establishes the final state immediately and does not promise a later event; DOCA_ERROR_IN_PROGRESS requires continued progress until completion.

Function-Level Reset

Trigger and registration: a host PCI function-level reset is reported through doca_devemu_pci_dev_event_flr_register(), registered on the PCI device obtained from doca_devemu_nvme_dev_as_pci_dev().

FLR is host-driven and differs from controller disable through CC.EN. It invalidates the function's existing emulation resources. Do not reuse old queue objects as though the host had merely paused I/O.

  1. Quiesce backend submissions for the affected function and account for application-owned requests and data movements.

  2. Follow Queue Draining and Teardown to drain affected work and remove the old queue bindings and queue objects. Handle DMA failures caused by the host reset without affecting unrelated controllers.

  3. Stop the NVMe device context, wait for Idle, then start it again to create fresh emulation resources.

  4. Wait for the host to reconfigure the controller and recreate its queues; do not re-enable the old host queue configuration.

Coordinate reset cleanup outside callbacks where necessary to avoid reentrant progress. Isolate application DMA resources per controller or recreate a DMA queue invalidated by reset, as described in Known Issues.

Completion condition: reset cleanup is complete and the device is ready for a fresh host controller-enable sequence. Host queue configuration from before FLR must not be re-enabled.

PCI Hotplug State Changes

Trigger and registration: a dynamic function changes PCI hotplug state. Register doca_devemu_pci_dev_event_hotplug_state_change_register() on the converted PCI device while its context is Idle.

  • Attach: prepare controller resources and event handling before making the function visible. Track the reported hotplug state, then handle the host's controller initialization through Controller Register Changes.

  • Detach: coordinate host removal and quiesce affected work. Use Queue Draining and Teardown for resource cleanup, respecting the PCI lifecycle before destroying the function.

Completion condition: the function has reached the intended attached or detached state. Do not treat an in-progress transition as complete. Host visibility and NVMe controller readiness are separate states; a hotplug event does not replace the NVMe enable handshake.

See DOCA DevEmu PCI for shared PCI hotplug mechanics and function-lifecycle requirements.

Queue Draining and Teardown

This section provides the shared procedures used by controller disable, Admin queue deletion, PCI FLR, detach and final teardown. Select only the stages required by the triggering operation: disabling processing does not by itself destroy resources or stop the application.

Disabling and Draining Processing

  1. Coordinate the affected controller or queue transition with the application's backend and stop initiating new work that conflicts with it.

  2. For a device-wide disable, call doca_devemu_nvme_dev_disable(). It cascades through SQs before CQs. For an individual queue, use its corresponding disable API.

  3. Check the return value. DOCA_SUCCESS establishes Disabled synchronously; DOCA_ERROR_IN_PROGRESS requires continued progression of the device and affected I/O contexts.

  4. Continue handling already-fetched request notifications and finish application-owned requests and payload transfers. Wait for the affected objects to reach Disabled before unbinding or destroying them.

  5. Resume the triggering controller or queue lifecycle flow. Remove resources only when that flow requires it; reserve complete teardown for final resource release.

Removing a Queue Pair

  1. Disable the SQ using doca_devemu_nvme_sq_disable(). DOCA_SUCCESS means it is disabled; DOCA_ERROR_IN_PROGRESS means draining is still required.

  2. Keep progressing the owning I/O context and finish delivered requests. Pending already-fetched requests may still be delivered after disable begins. doca_devemu_nvme_sq_flush() delivers pending request events; it is not a substitute for completing the backend work or waiting for Disabled.

  3. Wait until the SQ is Disabled. Then unbind and destroy it. Its CQ must remain available until all SQ activity has drained.

  4. Disable the CQ and wait for Disabled before unbinding it. Destroy it only after every SQ referencing it has been destroyed.

Complete Teardown

  1. Stop accepting new application work and coordinate host/controller removal.

  2. Call doca_devemu_nvme_dev_disable() if enabled. Keep all affected engines progressing until the device is Disabled. The device-level operation cascades through SQs before CQs.

  3. Unbind and destroy SQs, then unbind and destroy CQs, using ordinary destruction flags for final teardown.

  4. Stop each I/O context only after all its queue bindings are removed; wait for Idle and destroy it.

  5. Stop the device DOCA context only after it is Disabled and has no associated queues; wait for Idle and destroy it.

  6. Close representor handles and release application-owned resources. A dynamic function can be destroyed only through its valid PCI visibility/lifecycle flow; do not confuse freeing a software device with removing the host-visible function.

  7. Call doca_devemu_nvme_teardown(), remove registered emulation managers with doca_devemu_nvme_rm_dev(), then close devices when no other user needs them.

Export, Handover and Same-Build Takeover

The library exposes device-state export and import for applications implementing coordinated handover or crash takeover. These APIs preserve emulation state; the application must also preserve or reconstruct its own controller configuration, queue inventory and backend request state.

  1. Before the first device start, configure an absolute shared-memory directory with doca_devemu_nvme_dev_set_shm_dir_path(). Use suitably protected memory-backed storage. Once a nonempty path has been used by a successful start, do not change it during the device lifetime.

  2. While started, obtain the opaque descriptor with doca_devemu_nvme_dev_export(). The library owns the returned pointer and invalidates it when the device stops. Copy it securely if it must outlive that pointer; do not free the library-owned pointer.

  3. Use doca_devemu_nvme_dev_create_started_from_export() with a matching PCI type and matching create-started attributes. Re-register the callbacks and restore application state required by the destination.

  4. Use LOCAL queue creation/destruction flags only when reconstructing or relinquishing existing host queue state, not when creating or deleting new host queues.

  5. Coordinate ownership so at most one device in an ownership-transfer pair is enabled. For cross-version crash takeover and handover restrictions, see limitations 1 and 2.

State Machines

Context State

Context state

NVMe device

I/O context

Idle

Configure handlers, queue-pair count and optional SHM path; start or destroy.

Register request handler, connect PE; start or destroy.

Starting

Progress until Running; do not treat accepted start as controller readiness.

Follow DOCA Core start completion before binding queues.

Running

Handle registers/events and manage operational state and queues.

Progress requests/completions and perform queue operations on the owner thread.

Stopping

Progress to Idle after Disabled and queue cleanup.

Only stop after removing every queue binding; progress to Idle before destruction.

Device and Queue Operational State

State

Meaning

Next action

Disabled

Device/queue is not processing new work. A created queue starts here.

Bind and enable in dependency order, or unbind/destroy when dependencies permit.

Enabled

Processing is permitted; enabled queues still require correct bindings and controller configuration.

Process work or begin disable.

Disabling

Previously fetched requests and completion-side activity are draining.

Progress and complete outstanding work; wait for Disabled before unbind/destruction.

Context Running and operational Enabled are different conditions. Enable CQs before their SQs; disable SQs before their CQs. A synchronous success establishes the final state immediately; an in-progress result requires polling or the registered asynchronous state-change event.

Limitations

  1. Cross-version crash takeover is unsupported. The recovering process must use the same DOCA DevEmu library build as the exporting process; using another build has undefined behavior and can lose or duplicate in-flight commands.

  2. Handover of NVMe emulation state across software versions is not supported in this release.

Known Issues

  • SQE memory may be overwritten: In this release, SQE memory presented to the application may be reused while the request is still in flight, contrary to the request-lifetime guarantee. As a temporary workaround, copy needed command fields during the request callback rather than retaining and rereading the SQE pointer.

  • PCI vendor ID after hotplug takeover: In the affected SDK build, the recovered device for a hotplug function created by the same application process may return zero from doca_devemu_pci_dev_get_vendor_id(). Applications that rebuild Identify Controller data from this value can consequently report a zero VID after takeover. The function’s actual PCI configuration-space vendor ID remains unchanged. The reported reproduction retained working I/O; it did not demonstrate an I/O failure. This is a release-specific known issue, not the intended takeover behavior.

  • Unequal SQ/CQ depths: Use equal entry counts for each SQ and its CQ, including the Admin pair. Unsupported unequal depths can prevent queue/controller setup; recovery of a shallower SQ may restore the wrong head and replay or skip commands. The shallower-SQ recovery scenario was identified by code review and has not been reproduced.

  • Multiple SQs sharing one CQ: Creation of a second SQ against an already-used CQ fails. Allocate a dedicated CQ for each SQ.

  • Shared asynchronous-event descriptors: Selective progress mode cannot independently suspend contexts sharing an event descriptor. The progress thread may remain runnable and consume a CPU core. Do not assume notification requests will allow it to sleep.

  • FLR during in-flight DMA: A reset may leave an affected DMA queue unusable. Sharing it across controllers can affect unrelated controllers. Isolate DMA queues per controller or destroy and recreate the affected queue before resuming I/O.

API Reference

The public interface is declared in doca_devemu_nvme.h and the corresponding _type.h, _dev.h, _io.h, _cq.h and _sq.h headers. Use the generated API documentation matching the installed SDK version for complete signatures and return values.

API References

Reference Application

(3.6.0) DOCA Memos Initiator Application

Last updated: