SNAP Virtio-fs Service Release Notes

–

Description: The NFS fsdev module does not garbage collect its file objects. Once referenced, internal file objects persist unless the associated file is deleted, leading to stale object accumulation. When the per-fsdev limit of 1 million objects is reached, new file creation and directory listing operations fail.

Workaround: N/A

Keywords: File deletion/creation

Discovered in version: 1.4.0-doca3.1.0

–

Description: For hotplugged devices, mounting the device immediately after loading the driver may fail.

Workaround: Introduce a minimum delay of 3 seconds between driver load and mount operations to ensure proper device readiness.

Keywords: Wrong fs type; bad option; bad superblock; tag not found

Discovered in version: 1.4.0-doca3.1.0

–

Description: Repeatedly running virtio_fs_doca_device_hotplug and virtio_fs_doca_device_hotunplug RPCs in a loop can lead to recovery-related issues.

Workaround: Insert a minimum delay of 2 seconds between the hotplug and hotunplug operations to ensure system stability.

Keywords: Recovery failed, DOCA error

Discovered in version: 1.4.0-doca3.1.0

–

Description: The maximum supported scale for virtio-blk is 218 controllers.

Workaround: N/A

Keywords: Virtio-blk

Discovered in version: 1.3.0-doca3.0.0

–

Description: Invoking the virtio_fs_device_hotunplug RPC while the device is still mounted on the host, followed by a virtio_fs_device_hotplug call, may result in I/O errors on the mount point.

Workaround: Before issuing the virtio_fs_device_hotunplug RPC, ensure that the emulated file system is no longer mounted on the host. Use the umount command to cleanly unmount the device’s mount point.

Keywords: Hotunplug; mount

Discovered in version: 1.3.0-doca3.0.0

4346308

Description: Under high-scale scenarios, the virtio_fs_doca_manager_destroy RPC may time out.

Workaround: Run the RPC with an extended timeout value. For example: rpc.py -t 120  virtio_fs_doca_manager_destroy -m mlx5_0.

Keywords: Timeout

Discovered in version: 1.3.0-doca3.0.0

–

Description: The Linux Kernel NFS RDMA server is known to handle incoming connections slowly. At high scale (e.g., when using multiple hotplug functions), this can result in connection timeouts.

Workaround: Introduce a delay between mount operations of different hotplug functions on the host to avoid overwhelming the NFS RDMA server.

Keywords: Timeout, hotplug, mount

Discovered in version: 1.3.0-doca3.0.0

4422332

Description: The combination of virtio-fs, virtio-blk, and InfiniBand in a single use case is not supported.

Workaround: N/A

Keywords: IB

Discovered in version: 1.3.0-doca3.0.0

–

Description: I/O operations larger than 255 KB are not supported due to a data pool size limitation in the VirtioFS process running on the DPU.

Workaround: Set the max_xfer_size parameter in the fsdev_aio_create RPC to a value less than 255 KB. This parameter is used as max_write in the FUSE layer of the host kernel. If an I/O operation exceeds max_xfer_size, the FUSE layer automatically splits the request into smaller chunks.

The default max_xfer_size value is 128 KB. For example, if an I/O request is 1 MB and max_xfer_size is set to 64 KB, the FUSE layer on the host splits the operation into 16 separate 64 KB chunks, which are then individually handled by the virtio-fs process on the DPU.

Keywords: IO error; host error; io_size

Discovered in version: 1.2.0-doca2.10.0

–

Description: Due to the lack of recovery support, negative or resilience operations (e.g., destroy and restore) are not supported during active I/O traffic.

Workaround:  N/A

Keywords: Recovery; negative/resilience operations

Discovered in version: 1.0.0-doca2.8.0

4862565

Description: The following FUSE commands are unsupported: BMAP, SETUPMAPPING, REMOVEMAPPING, STATX

Workaround: N/A

Keywords: FUSE

Discovered in version: 1.0.0-doca2.8.0

 –

Description: Application restart is not supported if the application controller has already processed FUSE commands.

Workaround: Unload the virtio-fs driver on the host before restarting the application.

Keywords: FUSE

Discovered in version: 1.0.0-doca2.8.0

 –

Description: The following operations are not supported when using Linux's virtio-fs inbox/upstream kernel driver: FLR and the virtio-fs notification queue.

Workaround: N/A

Keywords: FLR; virtio-fs; inbox/upstream kernel driver

Discovered in version: 1.0.0-doca2.8.0

4687235

Description: Driver load on host or "echo <count> > /sys/bus/pci/devices/0000\:07\:00.2/sriov_numvfs" can get stuck forever due to resource constraint in DPU

Workaround: Restart the virtio_fs_tgt application

Keyworks: flexio, process stuck

Discovered in version: 1.5.0-doca3.2.0

4692130

Description: Triggering an FLR using echo 1 > /sys/bus/pci/devices/0000:07:00.2/reset on a PF with active VFs causes the VFs to become unusable.

Workaround: N/A

Keywords: FLR, VF

Discovered in version: 1.5.0-doca3.2.0

4552213

Description: Certain file operation sequences when using an aio fsdev backed by a Linux kernel NFS mount may result in silly-renamed files (with .nfs prefix) that are not automaticall removed.

Workaround:  They can be removed by dropping caches (echo 3 > /proc/sys/vm/drop_caches) from the host.

Keywords: AIO

Discovered in version: 1.5.0-doca3.2.0

4961367

Description: Enabling ASAN in debug mode while using the source package can lead to a heap-use-after-free issue.

Workaround: N/A

Keywords: source package, ASAN, debug

Discovered in version: 1.7.0-doca3.4.0

4990959

Description: Recovery may fail if the process is terminated before shared memory (shm) initialization completes.

Workaround: N/A

Keywords: Recovery

Discovered in version: 1.7.0-doca3.4.0

5000400

Description: Repeated events such as driver load/unload or device hotplug/hot-unplug that create and destroy virtqueues can eventually lead to resource exhaustion, causing virtqueue failures. This occurs when the process is running more than 1 ARM core.

Workaround: Restart the virtiofs process

Keywords: resource exhaustion, virtqueue fail

Discovered in version: 1.7.0-doca3.4.0

–

Description: Triggering a Function Level Reset (FLR) from the host by writing 1 to the reset file under /sys/bus/pci/devices/<bdf>/ (e.g., echo 1 > /sys/bus/pci/devices/0000:29:00.2/reset) causes the host driver to skip virtqueue creation. As a result, the mount point becomes unresponsive.

Workaround: Avoid initiating FLR while a virtio-fs mount is active on the host. To resume I/O after FLR, unload and reload the virtio-fs driver on the host.

Keywords: Driver; FLR

Discovered in version: 1.0.0-doca2.8.0

–

Description: On the host, when the virtio-fs mount is idle (i.e., no I/O operations), the dmesg log is repeatedly filled with AppArmor DENIED messages. These messages indicate that the ntpd service is being denied access to specific files, such as /snap/bin/ and /etc/ssl/openssl.cnf, due to restrictions in its AppArmor profile.

Workaround: Modify the AppArmor profile for ntpd to grant the necessary read permissions.

1. Locate the AppArmor profile for ntpd (typically located at /etc/apparmor.d/usr.sbin.ntpd).

2. Edit the profile and add the permissions using a text editor. For example:Run: sudo nano /etc/apparmor.d/usr.sbin.ntpd Add the following lines to allow ntpd to read the necessary files:  /snap/bin/ r, /etc/ssl/openssl.cnf r,

3. Reload the AppArmor profile to apply the changes:

   sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.ntpd
   
   
   

Keywords: AppArmor, ntpd

Discovered in version: 1.0.0-doca2.8.0

3949202

Description: On kernels older than version 6.10, repeatedly loading and unloading the virtio_pci and virtiofs drivers (or mounting and unmounting filesystems using these drivers) can cause the drivers to hang or become unresponsive.

Workaround: Insert a delay of at least 1 second between load and unload operations to reduce the risk of driver instability.

Keywords: Virtio_pci; virtiofs

Discovered in version: 1.0.0-doca2.8.0

-

Description: FUSE_STATX was introduced in FUSE version 7.39. If the negotiated version is less than this, kernel can still send FUSE_STATX.

Workaround: N/A

Keywords: Unsupported opcode

Discovered in version: 1.4.0-doca3.1.0

-

Description: On modern Linux distributions, unplugging a PCIe function from the host while there are inflight I/Os can cause the virtiofs driver to hang and virtio_fs_doca_device_hotunplug RPC will timeout

Workaround: N/A

Keywords: Hotplug, Linux

Discovered in version: 1.5.0-doca3.2.0