This library is supported at alpha level; backward compatibility is not guaranteed.
Introduction
The DOCA TLP DevEmu Virtio-blk library is part of the DOCA DevEmu Virtio subsystem. It provides low-level APIs for creating and managing virtio-blk devices through the device-emulation capabilities of NVIDIA® BlueField® DPUs.
Architecture and Library Interaction
This library is part of a modular subsystem, where each library manages its own logic:
-
DOCA DevEmu PCI – Manages the PCIe transport layer, including configuration, discovery, and features like MSI-X and device hot plug/unplug.
-
DOCA DevEmu Virtio Common – Manages generic virtio logic, such as handling common virtio registers, the virtio offload engine, and virtqueue lifecycle. Note: for TLP-based virtio-blk devices, the virtio device reset flow is handled by the application.
-
DOCA DevEmu Virtio-blk – Manages the specific logic for the virtio block device, including block I/O request parsing, scatter-gather segment handling, and block device configuration.
Each library manages a separate layer of the emulation stack.
Core Functionality
The library converts virtio descriptors that contain block I/O requests from the device driver into doca_devemu_vblk_req requests and routes them to the application through registered event handlers. Each request contains the block operation, target logical block address (LBA), and a DOCA buffer that represents the host-memory data region. The library abstracts the device-specific acceleration details.
The library supports two data path providers:
-
DPA (Data Path Accelerator) — Offloads virtqueue processing to BlueField's DPA for hardware-accelerated, low-latency operation. Recommended for production workloads.
-
DPU (Arm cores) — Processes virtqueues on the DPU Arm cores for maximum flexibility and debugging.
User Responsibilities
An application that uses this library must implement a virtio-blk controller that performs the following tasks:
-
Manage the underlying DOCA DevEmu Virtio-blk offload engine and I/O contexts.
-
Receive the abstracted DOCA Virtio-blk requests via registered event handlers.
-
Process these requests according to the virtio-blk specification (read, write, get device ID, etc.).
-
Perform direct memory access (DMA) transfers between host and DPU memory as needed.
-
Complete each request back to the I/O context with the appropriate status code.
The backend block storage implementation is outside the scope of DOCA. The reference application (doca_vblk_pci_dev) demonstrates a RAM-backed block device for illustration.
Prerequisites
Before you use this library, read the following guides:
Environment
DOCA DevEmu Virtio-blk is supported on the BlueField target only. The BlueField must meet the following requirements:
-
DOCA version 3.4.0 or later
-
BlueField-3 firmware 32.49.0288 or later
Note: See the DOCA Compatibility Policy. -
Run applications that use this library with root privileges.
Perform the following steps:
-
Configure BlueField to work in DPU mode as described in BlueField Modes of Operation.
-
Enable TLP emulation by running the following on the DPU:
[dpu] mlxconfig -y -d /dev/mst/mt41692_pciconf0 set TLP_EMULATION_ENABLE=1 [dpu] mlxconfig -y -d /dev/mst/mt41692_pciconf0 set TLP_EMULATION_NUM_PF=32 -
Perform a BlueField system cold reboot for the
mlxconfigsettings to take effect.
Hot-plug
Host Configuration
With a Linux environment on the host OS, additional kernel boot parameters are required to support the hot-plug feature:
-
For Intel machines:
intel_iommu=on iommu=pt pci=realloc -
For AMD machines:
iommu=pt pci=realloc
Note: On AMD machines, hotplug may not work.
Firmware Configuration
When PCIe switch emulation is enabled, BlueField can support hot-plugged virtio-blk functions. These PCIe functions are shared among all BlueField users and applications and may hold hot-plugged devices of type NVMe, virtio-blk, virtio-fs, or other (for example, virtio-net).
To enable PCIe switch emulation and configure hot-plugged ports, run:
[dpu] mlxconfig -y -d /dev/mst/mt41692_pciconf0 set PCI_SWITCH_EMULATION_ENABLE=1
[dpu] mlxconfig -y -d /dev/mst/mt41692_pciconf0 set PCI_SWITCH_EMULATION_NUM_TLP_PORT=1
[dpu] mlxconfig -y -d /dev/mst/mt41692_pciconf0 set PCI_SWITCH_EMULATION_NUM_PORT=2
PCI_SWITCH_EMULATION_NUM_PORT equals 1 plus the number of hot-plugged PCIe functions.
Architecture
The DOCA DevEmu Virtio-blk library provides the following main software abstractions:
-
The virtio-blk type – extends the PCIe TLP type; creates a PCI type specifically configured for virtio-blk device emulation
-
The virtio-blk offload engine – extends the virtio offload engine; manages virtqueue lifecycle and block-specific configuration (for example,
seg_max) -
The virtio-blk I/O context – extends the virtio I/O context; represents a progress context responsible for processing virtio descriptors carrying block I/O requests
-
The virtio-blk request – represents a single block I/O operation from the host driver
The following diagram illustrates the relationships between these abstractions:
Virtio-blk Feature Bits
The following virtio feature bits are relevant to virtio-blk emulation:
|
Feature Bit |
Description |
|---|---|
|
|
Maximum segment size is reported |
|
|
Maximum segments per request is reported |
|
|
Multi-queue support |
|
|
Indirect descriptor support |
|
|
VirtIO 1.0 compliance |
|
|
Device access restricted to platform IOMMU |
For TLP-based virtio-blk devices, feature bits are configured directly by the application in the emulated PCI configuration space. Indirect descriptor support must first be verified via doca_devemu_vblk_cap_is_indir_descs_supported() before enabling.
Virtio-blk Configuration Layout
According to the virtio specification, the virtio-blk device configuration structure layout includes the following key fields:
struct virtio_blk_config {
le64 capacity; /* device capacity in 512-byte sectors */
le32 size_max; /* maximum size of any single segment */
le32 seg_max; /* maximum number of segments in a request */
/* ... additional fields ... */
le16 num_queues; /* number of request queues */
/* ... */
};
For TLP-based virtio-blk devices, the application manages these fields directly in the emulated PCI configuration space. Configure seg_max and num_queues on the offload engine by calling doca_devemu_vblk_offload_engine_set_seg_max() and doca_devemu_virtio_offload_engine_set_num_queues(), respectively, so that the library can allocate the required resources.
Virtio-blk Type
The virtio-blk type creates a PCI TLP type specifically configured for virtio-blk device emulation, providing full control over the PCIe configuration space.
To create and configure a virtio-blk type:
-
doca_devemu_vblk_is_pci_tlp_type_supported(): Check if the device supports virtio-blk TLP type management. -
doca_devemu_vblk_pci_tlp_type_create(): Create a stopped PCI TLP type for virtio-blk emulation. -
Configure the type using the following setters:
-
doca_devemu_pci_type_set_dev(): Associate a DOCA device with the type. -
doca_devemu_pci_tlp_type_set_pci_cap_conf(): Register PCI capability configurations (for example, PCI Express, MSI-X). -
doca_devemu_pci_type_set_num_msix(): Set the number of MSI-X vectors. -
doca_devemu_pci_type_set_num_db(): Set the number of doorbells.
-
-
doca_devemu_pci_type_start(): Start the PCI type to finalize configuration.
Virtio-blk Offload Engine
The virtio-blk offload engine (doca_devemu_vblk_offload_engine) extends the virtio offload engine with block-specific configuration. It is responsible for managing the state of all offloaded entities on a device, including configuration, creation, destruction, start, and stop of virtio-blk request virtqueues. Each offload engine is associated with a single DOCA DevEmu PCI endpoint.
Offload Engine Lifecycle
The offload engine follows this lifecycle. Configuration (for example, seg_max, indirect descriptors) happens between create() and start(). In handover and recovery scenarios, create_from_export() replaces both create() and the configuration phase — configuration values are inherited from the export descriptor:
Standard Configuration
The standard flow for creating and configuring a virtio-blk offload engine is as follows:
-
doca_devemu_vblk_offload_engine_create(): Create an offload engine bound to a PCI endpoint. -
doca_devemu_vblk_offload_engine_set_seg_max(): Set the maximum scatter-gather segments per request. -
Cast to virtio offload engine via
doca_devemu_vblk_offload_engine_as_virtio_offload()for common configuration:-
doca_devemu_virtio_offload_engine_set_indir_descs_enabled(): Enable/disable indirect descriptor processing. -
doca_devemu_virtio_offload_engine_set_num_queues(): Set the number of virtqueues.
-
-
doca_devemu_virtio_offload_engine_start(): Start the offload engine. -
doca_devemu_virtio_offload_engine_enable(): Enable the offload engine to begin processing requests.
Optional Configurations
-
doca_devemu_vblk_offload_engine_set_seg_max(): Set theseg_maxvalue for the block device. If not set, the value defaults to the globalseg_maxconfigured viadoca_devemu_vblk_set_seg_max(). -
doca_devemu_virtio_offload_engine_set_indir_descs_enabled(): Enable indirect descriptor support. Only valid ifdoca_devemu_vblk_cap_is_indir_descs_supported()returns true.
Export/Import Overlay
|
Function |
When to Call |
Effect |
|---|---|---|
|
|
After |
Produces an opaque export descriptor containing the runtime state of the offload engine. The caller owns the descriptor and must free it with |
|
|
Instead of |
Creates a new idle offload engine seeded with the state from an export descriptor. The new engine must still be started and enabled. |
Offload Engine States
A started offload engine is in one of three states:
|
State |
Description |
|---|---|
|
|
Initial state after |
|
|
Processing operations from the host driver. Virtqueues can be individually enabled/disabled. |
|
|
Transitioning to |
Enable/Disable Semantics
-
enable()moves all started-and-disabled virtqueues to the enabled state. If the engine is already enabled, callingenable()again enables any newly started virtqueues that are currently disabled. -
disable()may returnDOCA_SUCCESS(synchronous) orDOCA_ERROR_IN_PROGRESS(asynchronous, at least one VQ is still draining). In the async case, poll withget_state()until the engine reachesDISABLED.
Handover Constraint
During a handover, for each source/destination pair of offload engines sharing the same PCI endpoint, at most one engine may be enabled at any time. Both engines may be disabled simultaneously (between the source's disable() and the destination's enable()), but having both enabled leads to undefined behavior.
State Export/Import (Live Upgrade and Recovery)
The DOCA DevEmu Virtio Block SDK provides a state export/import mechanism that enables two resilience scenarios for an offload engine managing a Virtio Block device:
|
Scenario |
Description |
|---|---|
|
Handover (Live Upgrade) |
A running source process exports the offload engine state and transfers it to a destination process. The destination imports the state, and the two processes coordinate so that exactly one engine is enabled at any time. In-flight I/O during handover is not supported. |
|
Takeover (Recovery) |
A running process exports the offload engine state and stores the export descriptor in a location outside its own process that a recovering process can access. If the process crashes, a recovering process imports the descriptor to recreate the offload engine and replay in-flight I/O. |
Both scenarios use the same three API calls:
doca_devemu_vblk_offload_engine_export()
doca_devemu_vblk_offload_engine_create_from_export()
doca_devemu_vblk_offload_engine_export_release()
API Reference
|
API |
Description |
Preconditions |
Postconditions |
|---|---|---|---|
|
|
Exports the runtime state of a started offload engine into an opaque descriptor blob. |
|
|
|
|
Creates a new offload engine from an export descriptor. This is the import-side counterpart of |
|
|
|
|
Releases engine-side resources associated with the export. Must be called exactly once per export. Does not free the descriptor buffer — the caller is responsible for freeing it with |
|
N/A |
Handover (Live Upgrade) Flow
In a handover scenario, a source process transfers ownership of an offload engine to a destination process while maintaining host-visible state. The SDK does not prescribe any particular IPC mechanism or process architecture - the coordination protocol is application-defined.
-
Source has a valid export descriptor. The
export()call is typically done during regular bring-up of the application, not necessarily at the time of handover. -
Source transfers the export descriptor to the destination through an application-defined channel.
-
Destination calls
create_from_export()with the received descriptor and a PCI endpoint handle. This creates an idle offload engine seeded with the source's state. -
Destination calls
start()on the new engine. Configuration is inherited from the descriptor. The engine enters theDISABLEDstate. -
Destination calls
export()on its own engine to prepare for future recovery or live update. Virtqueues must not be started before this call. -
Destination creates, configures, and starts its virtqueues.
-
Source calls
disable()on its engine. IfDOCA_ERROR_IN_PROGRESSis returned, pollget_state()untilDISABLED. -
Once the source is fully disabled, destination calls
enable()on its engine. The engine and its virtqueues must be enabled to start processing I/Os from the host driver. -
Source calls
stop()anddestroy()to release its engine. -
export_release()is called on the source's descriptor.
Key invariant
At no point during the handover should both engines be enabled simultaneously.
Takeover (Recovery) Flow
In a takeover scenario, a process exports its offload engine state proactively so that a recovering process can restore the engine after a crash.
-
Process A calls
export()on its started offload engine and stores the export descriptor in a location outside its own process that a recovering process can access (for example, shared memory, file, or another process). -
Process A crashes.
-
Process B (the recovering process) retrieves the stored export descriptor.
-
Process B calls
create_from_export()with the descriptor and a PCI endpoint handle. This creates an idle offload engine seeded with the crashed process's state. -
Process B calls
start()on the engine. -
Process B creates, configures, and starts its virtqueues, then calls
enable()on the engine. The engine and its virtqueues must be enabled to start processing I/Os. In-flight I/O from the recovery state is replayed.
Process B must configure the virtio-blk global parameters (for example, doca_devemu_vblk_set_*()) with the same values that were used by Process A before takeover. These values are not captured in the export descriptor.
Request Virtqueue Management
Each virtio-blk virtqueue is created and managed through the offload engine:
|
API |
Description |
|---|---|
|
|
Create a virtio-blk request virtqueue from a started offload engine. Must be called on the offload engine thread. |
|
|
Destroy a virtio-blk request virtqueue. Must not be started. |
|
|
Cast to base virtio virtqueue for common operations ( |
Virtio-blk I/O Context
The virtio-blk I/O context (doca_devemu_vblk_io) extends the virtio I/O context. It is responsible for routing incoming block I/O requests from the host driver towards the application, and relaying completions back to the driver. Each I/O context is associated with a single offload engine and is progressed by a single DOCA Core progress engine. Usually, users configure a single virtio-blk I/O context per BlueField Arm core used by the application.
The I/O context can only route requests while in running state and when its associated offload engine is enabled.
Virtio-blk I/O Configurations
The flow for creating and configuring a virtio-blk I/O context is as follows:
-
doca_devemu_vblk_io_create_from_offload_engine(): Create a new I/O context from a started offload engine. Must be called on the Arm core that will manage the I/O context. -
doca_devemu_vblk_io_event_vblk_req_register(): Register the event handler for incoming virtio-blk requests. -
Cast to DOCA context via
doca_devemu_vblk_io_as_ctx()for progress engine connection:-
doca_pe_connect_ctx(): Connect the I/O context to a progress engine. -
doca_ctx_start(): Start the I/O context.
-
Mandatory Configurations
-
doca_devemu_vblk_io_event_vblk_req_register(): Register the event handler for incoming virtio-blk requests. This is mandatory.
Virtio-blk Request
The virtio-blk request object (doca_devemu_vblk_req) serves as an abstraction for handling block I/O requests arriving on virtio-blk request queues. These requests are generated by the device driver through virtio queues and routed to the user via the registered event handler on the associated I/O context.
Once the event handler is called, ownership of the virtio-blk request and the associated request user data transfers to the user. The ownership moves back to the associated I/O context when the request is completed by calling doca_devemu_vblk_req_complete().
The event handler provides the following parameters directly:
void handler(struct doca_devemu_vblk_req *req,
uint32_t type, /* VIRTIO_BLK_T_IN, VIRTIO_BLK_T_OUT, etc. */
uint64_t sector, /* starting sector (LBA) */
void *req_user_data); /* per-request user data buffer */
The following APIs operate on a virtio-blk request:
|
API |
Description |
|---|---|
|
|
Get a DOCA buffer representing the |
|
|
Get the total data length (in bytes) of the data buffer linked list. |
|
|
Get the number of scatter-gather elements in the data buffer linked list. |
|
|
Get the associated DOCA Virtio Block I/O context. |
|
|
Get the user data associated with the virtqueue binding. Must be called on the I/O context thread. |
|
|
Complete the request with a status value and the number of bytes written into the device-writable portion of the buffer. The status follows the virtio specification: |
Queue Counters
Each virtio-blk request virtqueue maintains a set of cumulative per-queue counters that cover generic virtio activity and block-specific data path events. The application reads them through a counters list object created on the offload engine.
Available Counters
The following counters are maintained per virtqueue:
|
Counter |
Description |
|---|---|
|
|
Number of descriptor-chain heads fetched from the available ring |
|
|
Number of descriptor-chain heads completed towards the driver |
|
|
Number of doorbell events handled for the virtqueue |
|
|
Number of MSI-X interrupts raised for the virtqueue |
|
|
Number of requests retried because the shared |
|
|
Number of requests completed with a status other than |
|
|
Number of inbound DMA failures while fetching the request header or the indirect descriptor table |
|
|
Number of outbound DMA failures while writing the status byte back to the host |
The first four are common virtio counters, shared with other virtio device types. Read them with doca_devemu_virtio_queue_counters_get_<counter>() after casting the list element with doca_devemu_vblk_queue_counters_as_virtio_queue_counters().
The last four are virtio-blk specific. Read them with doca_devemu_vblk_queue_counters_get_<counter>() directly on the list element.
Every getter returns DOCA_ERROR_NOT_SUPPORTED if the virtqueue does not maintain that counter, and DOCA_ERROR_BAD_STATE if the element has not been populated.
Reading the Counters
Perform the following steps:
-
doca_devemu_vblk_offload_engine_create_queue_counters_list(): Create a counters list on an enabled offload engine. The list keeps the offload engine enabled for as long as it exists. -
doca_devemu_vblk_queue_counters_populate_list(): Start an asynchronous population. Only one population may be in progress per offload engine. -
doca_devemu_vblk_queue_counters_is_populated(): Poll until the list reports that the population has finished. -
Read the values with the getters described above. Elements are addressed as
(*counters_list)[queue_index]. -
doca_devemu_vblk_queue_counters_destroy_list(): Destroy the counters list. Steps 2 to 4 may be repeated before the list is destroyed.
Each element holds the values captured by a single population, so the application reads a stable sample instead of accumulators that the data path keeps advancing.
In DPA provider mode, populating the four common counters requires a round trip to the device, because the DPA owns the data path that advances them. In DPU provider mode, and for any virtqueue that is not enabled, the values are sampled in place and the population completes without waiting.
Note: The counters are cumulative and there is no reset operation. They are stored in the virtqueue object, so a virtqueue that is destroyed and recreated, for example by a host driver reload, a PCI bind cycle, or an FLR, starts counting from zero. An application that compares samples across such an event must take a new baseline.
Counters List Management
|
API |
Description |
|---|---|
|
|
Create an unpopulated list of per-virtqueue counters for a started and enabled offload engine. Only a single counters list is allowed per offload engine. |
|
|
Issue an asynchronous population of the counters list. Returns |
|
|
Check whether the counters list has finished populating for all virtqueues. |
|
|
Cast a virtio-blk queue counters element to the common virtio queue counters, for use with the common getters. |
|
|
Get the virtqueue index of a populated element. |
|
|
Destroy the counters list. Returns |
Counter Capability Queries
|
API |
Description |
|---|---|
|
|
Check whether the device supports a given virtio-blk queue counter. |
|
|
Check whether the device supports a given common virtio queue counter. |
Global Configuration and Capability Queries
Before initializing the virtio-blk subsystem, the application must configure global parameters and may query device capabilities.
Global Configuration
The following APIs manage global configuration and the subsystem lifecycle. Call the configuration setters before doca_devemu_vblk_init(). For the setter APIs (set_vblk_req_user_data_size, set_max_queue_size, set_seg_max, set_datapath_on_dpa), the last value set before doca_devemu_vblk_init() takes effect.
|
API |
Description |
|---|---|
|
|
Register a DOCA device for virtio-blk management. The device must support virtio-blk type management. |
|
|
Unregister a previously added DOCA device. |
|
|
Set the size of the per-request user data buffer allocated for each |
|
|
Set the maximum virtqueue depth. Must conform to |
|
|
Set the global maximum scatter-gather segments per request. Must conform to |
|
|
Set to |
|
|
Initialize the virtio-blk subsystem. One-time call after all configuration is complete. |
|
|
Tear down the virtio-blk subsystem and release all global resources. |
Capability Queries
|
API |
Description |
|---|---|
|
|
Query the maximum supported virtqueue depth for the device. |
|
|
Query the maximum supported |
|
|
Check whether the device supports indirect descriptor processing. |
Discovery
The virtio-blk library uses the TLP emulation path, creating its own PCI type and representors dynamically.
To find and prepare a suitable device for virtio-blk emulation, perform the following steps:
-
Open a DOCA device that supports virtio-blk TLP type management. This can be done by:
-
Iterating
doca_devinfo_create_list()and checkingdoca_devemu_vblk_is_pci_tlp_type_supported()for each device, or -
Opening a known device directly by IB device name (for example,
mlx5_0) via helper utilities such asopen_doca_device_with_ibdev_name().
-
-
doca_devemu_vblk_pci_tlp_type_create(): Create the virtio-blk PCI TLP type. -
Configure and start the PCI type (
doca_devemu_pci_type_set_dev(),doca_devemu_pci_type_start(), etc.). -
doca_devemu_pci_type_create_rep(): Create representors for each endpoint. -
doca_devemu_pci_tlp_dev_create(): Create PCI TLP devices from the representors.
At this point, the user can proceed with the initialization flow to create offload engines and I/O contexts.
Initialization
This section describes the initialization flow of a DOCA DevEmu Virtio-blk offload engine and one or more I/O contexts. During initialization, the user sets up and prepares the environment before starting to receive block I/O requests from the host.
Perform the following steps:
-
Configure global virtio-blk parameters (
add_dev,set_seg_max,set_datapath_on_dpa, etc.) and calldoca_devemu_vblk_init(). -
Choose N Arm cores to run the application I/O threads on.
-
Create N DOCA Core progress engine (PE) objects.
-
Create the virtio-blk PCI TLP type via
doca_devemu_vblk_pci_tlp_type_create(), configure capabilities, and start it. -
Create a PCI TLP device for the endpoint.
-
Configure global offload engine parameters.
-
Create and configure the virtio-blk offload engine:
-
If offload engine doesn't perform state import:
-
doca_devemu_vblk_offload_engine_create()with the PCI endpoint. -
Configure
seg_max,num_queues, and optionally indirect descriptors. -
doca_devemu_virtio_offload_engine_start(). -
Optionally, call
doca_vblk_pci_export()for future handover/export (see State Export/Import (Live Upgrade and Recovery)).
-
-
If state import:
-
doca_devemu_vblk_offload_engine_create_from_export()for future export, passing the export descriptor and its length along with the PCI endpoint. -
If offload engine is a handover destination, call
doca_devemu_vblk_offload_engine_export(). -
doca_devemu_virtio_offload_engine_start().
-
-
-
Create N virtio-blk I/O contexts:
-
doca_devemu_vblk_io_create_from_offload_engine()on each Arm core. -
Register the block request event handler.
-
Connect to the progress engine and start.
-
-
Enable the offload engine via
doca_devemu_virtio_offload_engine_enable().
At this point, the offload engine and I/O contexts are fully operational and ready to process block I/O requests from the host.
During the initialization flow, no block I/O requests are routed to the user until the offload engine is enabled.
Teardown
This section describes the teardown flow of a DOCA DevEmu Virtio-blk offload engine and its associated I/O contexts.
Perform the following steps:
-
For each bound virtqueue, perform the per-VQ teardown sequence:
-
doca_devemu_virtio_vq_disable(): Disable the virtqueue. -
If the call returns
DOCA_ERROR_IN_PROGRESS(virtqueue is draining in-flight requests), optionally calldoca_devemu_virtio_io_flush_vq()to flush pending events to the application. -
Handle and complete any outstanding virtio-blk requests via
doca_devemu_vblk_req_complete(). -
Loop: call
doca_pe_progress()on the I/O context's progress engine until the virtqueue reachesDISABLEDstate (poll viadoca_devemu_virtio_vq_get_state()). -
doca_devemu_virtio_io_unbind_vq(): Unbind the virtqueue from the I/O context.
-
-
For each virtio-blk I/O context:
-
Stop the I/O context via
doca_ctx_stop()and wait foridlestate. -
Destroy the I/O context via
doca_devemu_vblk_io_destroy().
-
-
Disable the offload engine via
doca_devemu_virtio_offload_engine_disable()after all virtqueues are disabled. -
Destroy all virtqueues via
doca_devemu_vblk_req_vq_destroy(). -
Stop and destroy the offload engine:
-
doca_devemu_virtio_offload_engine_stop() -
doca_devemu_vblk_offload_engine_destroy()
-
-
Stop and destroy the PCI TLP type and devices.
-
Tear down the virtio-blk subsystem via
doca_devemu_vblk_teardown().
Execution Phase
This section describes execution on BlueField Arm cores using DOCA Core progress engines.
Control Path
The DOCA Virtio-blk offload engine manages the control path, including:
-
Virtqueue Lifecycle: Creation, configuration, start, enable, disable, stop, and destruction of request virtqueues, driven by host driver state transitions (for example,
DRIVER_OK, device reset). -
Device Reset: When the host driver writes
device_status = 0, the virtio core library triggers a reset event. The application must tear down all active virtqueues, flush in-flight operations, and reinitialize the device. -
Offload Engine State: The engine transitions through
DISABLED → ENABLED → DISABLING → DISABLED, as described in Offload Engine States.
I/O Path
This section describes the flow for a single virtio-blk request sent by the device driver until its completion.
Before processing requests, register an event handler by calling doca_devemu_vblk_io_event_vblk_req_register().
The application must also understand the virtio-blk specification and be able to perform DMA operations between the host and local memory by using DOCA DMA or another suitable method.
The I/O path for a block request:
-
Request arrival – The DOCA library invokes the registered event handler with the request object, type, sector, and user data.
-
Request inspection – The application examines the request type:
-
VIRTIO_BLK_T_IN(0) – Read from block device. -
VIRTIO_BLK_T_OUT(1) – Write to block device. -
VIRTIO_BLK_T_GET_ID(8) – Return 20-byte device serial number. -
Other types (for example,
VIRTIO_BLK_T_FLUSH,VIRTIO_BLK_T_DISCARD) – The application decides how to handle these based on the feature bits it has exported to the host.
-
-
Data access – The application calls
doca_devemu_vblk_req_get_data()to obtain a DOCA buffer representing the host memory data region, anddoca_devemu_vblk_req_get_data_len()for the total data length. -
DMA transfer – For read operations, the application populates a local buffer and uses DOCA DMA to transfer data to host memory. For
VIRTIO_BLK_T_INandVIRTIO_BLK_T_GET_IDrequests, calldoca_buf_reset_data_len()for each DOCA buffer segment before transferring data to host memory. For write operations, the application uses DOCA DMA to transfer data from host memory to a local buffer. -
Request completion – The application calls
doca_devemu_vblk_req_complete(req, status, len)with the appropriate status code and the number of bytes written to the device-writable portion of the buffer.
Last updated: