DOCA Platform Framework v26.8.0

This is the GA release of the DOCA Platform Framework (DPF). It includes bug fixes and improvements to enhance the provisioning and orchestration of NVIDIA BlueField DPUs in Kubernetes environments.

Revision History

Features

• NVIDIA BlueField-4 support (Beta)

  • This is the first DPF release to support NVIDIA BlueField-4 DPUs, for selected partners and customers. NVIDIA BlueField-4 support is Beta in this release. NVIDIA BlueField-3 support continues to be Generally Available (GA).

  • Includes ASTRA, the E/W NIC zero-trust boundary (astraEnabled on DPUSet/DPUDeployment) that DOCA Weave builds on.

• DOCA Weave DPUService (Tech Preview)

  • DOCA Weave provides east-west network isolation on NVIDIA BlueField-4 DPUs with ConnectX-9 east-west NICs, using OVS on the DPU to enforce per-tenant isolation independently of the host and hardware-accelerated VXLAN encapsulation toward the E/W NICs. Deployed alongside DOCA XPlane, which handles hardware multi-plane failover across the E/W fabric.

  • For details, see DOCA Weave service.

• NodeServiceInterfaces API to replace per-node ServiceInterface objects to enable scalability

  • DPF introduces NodeServiceInterfaces, a per-node shard CR that replaces one legacy ServiceInterface object per interface per node, cutting SFC interface object count from one per interface to one per DPU — a scaling improvement that reduces API object count and load at scale (for example, from tens of thousands of objects to one per DPU in large clusters). Plain SFC interfaces use this path by default; VPC interfaces still use the legacy ServiceInterface path.

  • For details, see NodeServiceInterfaces.

• Zero Trust BMC and DPU Agent security hardening

  • DPF adds a DPF CA trust bundle distributed to DPUs, periodic rotation of BMC mTLS server certificates, per-DPU BMC credentials (DPUDevice.spec.bmcCredentialSecretName) instead of a single shared BMC password, and a one-time BMC factory reset during DPU onboarding so every managed account starts from a known, non-default password.

  • For details, see CA Certificate Rotation, Per-DPU BMC Credentials, and BMC Factory Reset.

• SPIFFE/SPIRE workload identity

  • DPF now supports SPIFFE-based identity. DPFOperatorConfig.spec.security.spiffe gives the DPU Agent its own SPIFFE identity for authenticating to the management-cluster API server, and DPUService.spec.security.spiffe (or DPUServiceTemplate.spec.security.spiffe) lets individual DPUServices opt in to their own SPIFFE identity.

  • For details, see SPIFFE DPU Agent Identity.

• Etcd encryption at rest for Kamaji DPU clusters, with OpenBao/Vault KMS integration

  • DPF can now encrypt Secrets and ConfigMaps in Kamaji-managed DPU cluster etcd via DPFOperatorConfig.spec.kamajiClusterManager.etcdEncryptionAtRest, using either a rotatable staticKey or a new vaultKMS KMS v2 plugin backed by OpenBao/HashiCorp Vault Transit.

  • For details, see Kamaji Encryption at Rest.

• Opt-in enforcement for privileged DPUService pods

  • Admins now get explicit control over whether privileged DPUServices can be deployed on DPUs. DPUServices that run privileged pods must explicitly opt in via DPUService.spec.security.privileged (or DPUServiceTemplate.spec.security.privileged).

  • For details, see DPUServiceSecurity.

• Kata Containers support for DPU workload isolation

  • DPF can now deploy Kata Containers to DPU clusters via DPFOperatorConfig.spec.security.kata, providing VM-based isolation for untrusted workloads. Disabled by default; workloads opt in with runtimeClassName: kata-qemu.

  • For details, see Kata Containers and Kata Containers on Host Workers.

• Native multi-DPUCluster IP allocation for DPUServiceIPAM

  • DPUServiceIPAM now natively supports targeting multiple DPUClusters via dpuClusterSelector, automatically computing per-cluster ExcludeRanges so IP allocations don't overlap across clusters.

  • For details, see DPUServiceIPAM and DPUCluster.

• Host boot gating on DPU Readiness

  • DPF can now hold the host at UEFI until the DPU is genuinely ready to serve traffic, closing the window where workloads land on a host before its DPU-backed networking and storage are actually up. This behavior is controlled via DPUFlavor.spec.serviceReadiness.gate, in conjunction with the DELAY_HOST_OS_INIT NVConfig hold.

  • For details, including important caveats, see Service Readiness.

• DPUFlavor enhancements: per-DPU templating and node labeling

  • DPF can now render a per-DPU DPUFlavor from a shared DPUFlavorTemplate using values from DPUDevice.spec.values, so device-specific configuration no longer requires one static DPUFlavor per DPU. Available via DPUSet.spec.dpuTemplate.spec.dpuFlavorTemplate and DPUDeployment.spec.dpus.flavorTemplate.

  • DPUFlavor.spec.configFiles can now deliver executable scripts that the DPU agent runs during provisioning to apply Kubernetes Node labels, letting operators attach hardware-discovered or environment-specific labels automatically to the corev1.Node that represents a DPU inside the DPUCluster.

  • For details, see DPUFlavorTemplate: Per-DPU Flavor Rendering and DPU Node Label Scripts.

• Observability improvements: lower metric cardinality and DPU workload metrics streaming

  • DPF-managed Prometheus scrape configurations now apply metric allowlists across the management cluster control plane, DPF controllers, kube-state-metrics, and DPU cluster control planes, substantially reducing Prometheus TSDB series count and its growth per DPUCluster.

  • The OpenTelemetry Collector on each DPU cluster now streams container, pod, and node resource metrics to the management cluster Prometheus, tagged by DPUCluster, alongside a new DPUService Performance Grafana dashboard.

  • For details, see DPF-Operator-Managed Components and Grafana Dashboards.

• DPF upgrade improvements: pre-upgrade validation

  • Before upgrading its managed components, the DPF operator now validates existing API objects against the OpenAPI schema of the installed CRDs and blocks the upgrade if violations are found.

  • For details, see Object Schema Validation.

• dpfctl diagnostics improvements

  • dpfctl dump bmc collects BMC diagnostic dumps, dpfctl sosreport gains configurable CPU/memory requests and limits, and dpfctl describe adds --issues and --show-pending flags plus shell completion for flag values.

  • For details, see the dpfctl docs.

Fixed Issues from Previous Release

• [OVN-Kubernetes DPUService] Node annotations are now correctly refreshed when the uplink PF IP allocation changes on DPU-enabled hosts

• Stale dpudevice-protection finalizers on non-selected DPUDevices no longer block cleanup after upgrading from v25.10

Supported DPU Services

N/A for DOCA Weave and DOCA XPlane means these services require ConnectX-9 SuperNICs and specific GPU platform support (e.g. NVIDIA Vera Rubin NVL72, NVIDIA HGX Rubin NVL8), available only with NVIDIA BlueField-4.

Kata Containers is a generic reference: any container (or Kata-isolated container) can run as a DPUService. See dummydpuservice in the DPUService development guide for a minimal example to build from.

Dependencies

Hardware and Software Requirements

• DPU Hardware:

  • NVIDIA BlueField-3 DPUs (GA)

  • NVIDIA BlueField-4 DPUs (Beta, selected partners and customers)

• Minimal DOCA BFB Image: DOCA v2.5 or higher (must be pre-installed on DPUs to support DPF provisioning)

• Supported DOCA BFB Image (BlueField-3):

  • bf-bundle-3.2.x

  • bf-bundle-3.4.0

  • bf-bundle-3.5.0

  • See the DPF Compatibility Policy for the full support window.

• Supported DOCA ISO and Firmware (BlueField-4): Available to selected partners and customers.

Installation & Upgrade Notes

Installation Notes

None.

Upgrade Notes

• DPFOperatorConfig.spec.deploymentMode is now required

  • Description:DPFOperatorConfig.spec.deploymentMode is a new required field with values zero-trust or host-trusted. It must be set explicitly; the API rejects an empty value. The operator validates that deploymentMode is consistent with provisioningController.installInterface before reconciling system components.

  • Impact: Clusters upgraded from a release without deploymentMode retain a DPFOperatorConfig that lacks the field. Reconciliation of system components (including the provisioning controller) will not proceed until the field is set. Existing DPUs and DPUServices continue running during this window.

  • Required action: After upgrading to v26.8.0, patch the existing DPFOperatorConfig to set deploymentMode. If your previous configuration used installViaRedfish, set deploymentMode to zero-trust. Host-trusted deployments must not use installViaRedfish; omit installInterface or use installViaHostAgent instead.

    Bash

    # Zero Trust (e.g. previously used installViaRedfish)
    kubectl patch dpfoperatorconfig dpfoperatorconfig -n dpf-operator-system --type merge -p '{"spec":{"deploymentMode":"zero-trust"}}'
    
    # Host Trusted
    kubectl patch dpfoperatorconfig dpfoperatorconfig -n dpf-operator-system --type merge -p '{"spec":{"deploymentMode":"host-trusted"}}'
    

  • Verification:

    Bash

    kubectl get dpfoperatorconfig dpfoperatorconfig -n dpf-operator-system -o jsonpath='{.spec.deploymentMode}'
    

• Existing privileged DPUServices without security.privileged set keep running, but enforcement will tighten in a future release

  • Description: New DPUServiceTemplate/DPUService objects must set security.privileged explicitly when deployInCluster is false; the API rejects the object otherwise. Existing DPUServices that already had security.privileged unset before upgrading to v26.8.0 are exempted from this validation and continue to be treated as privileged, so they keep running without disruption.

  • Impact: This exemption is planned to be removed in a future release, after which it will default to deny. DPUServices that rely on it will be blocked from running privileged pods once that happens.

  • Required action: Explicitly set security.privileged on any existing DPUServiceTemplate/DPUService that runs privileged pods, so they keep working once the exemption is removed.

• CoreDNS for DPU clusters now runs on the host cluster

  • Description: DPU cluster DNS is now served by CoreDNS running on the host cluster instead of inside the DPU cluster. This reduces per-DPU-cluster load and removes the dependency on DPU-to-DPU communication over the OOB network for DNS resolution, which is relevant for Zero Trust environments where that path is blocked. Configurable via DPFOperatorConfig.spec.coreDNS.

  • Impact: No manual action is required; DPU cluster DNS resolution continues to work after upgrade.

• SFC interfaces auto-migrate to NodeServiceInterfaces

  • Description: After upgrading to v26.8.0, SFC ServiceInterfaceSet resources (those without a virtualNetwork) are automatically moved onto the NodeServiceInterfaces path. On reconcile, the controller stamps svc.dpu.nvidia.com/interface-mode: nsi, creates the corresponding NodeServiceInterfaces entries, and deletes owned legacy per-node ServiceInterface children only after those NSI entries are Ready. No manual action is needed.

  • Impact:

    • Migration is make-before-break: NSI entries are created and must become Ready before legacy ServiceInterface objects are removed, so SFC traffic continuity is preserved during upgrade.

    • Operators and automation that inspect DPU-cluster interface objects should look for NodeServiceInterfaces in dpf-operator-system rather than per-node ServiceInterface objects for SFC.

    • VPC interfaces are not migrated in this release and continue to use ServiceInterface objects.

    • The legacy SFC ServiceInterface code path is planned for removal in a future release once migration has completed in the field.

  • Verification:

    Bash

    # Interface mode on DPU-cluster ServiceInterfaceSets (SFC should report nsi)
    kubectl get serviceinterfacesets -A \
      -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}{"\t"}{.metadata.annotations.svc\.dpu\.nvidia\.com/interface-mode}{"\n"}{end}'
    
    # Legacy per-node ServiceInterface objects should be gone for migrated SFC sets
    # (using DPUCluster kubeconfig)
    kubectl get serviceinterfaces -A
    
    # NSI path: per-node shards in the DPF operator system namespace on the DPU cluster
    kubectl get nodeserviceinterfaces -n dpf-operator-system
    

Deprecated API Fields

• DPUFlavor.spec.dpuMode

  • Description:DPUFlavor.spec.dpuMode is deprecated and no longer drives provisioning behavior. Deployment mode is configured on DPFOperatorConfig and exposed on DPU.status.deploymentMode. Existing flavors that set dpuMode continue to be accepted, but the field is ignored by provisioning workflows.

  • Deprecated field:

    • DPUFlavor.spec.dpuMode -> Use DPFOperatorConfig.spec.deploymentMode instead

• DPFOperatorConfig.spec.ovsCNI

  • Description:DPFOperatorConfig.spec.ovsCNI is deprecated. OVS CNI is now installed automatically by the CNIInstaller component instead of being deployed as a standalone chart.

  • Deprecated field:

    • DPFOperatorConfig.spec.ovsCNI -> Use DPFOperatorConfig.spec.cniInstaller instead

Known Issues and Limitations

• Socket Direct environments are not supported

  • DPF does not currently support environments where NVIDIA Mellanox Socket Direct adapters are used. Socket Direct is a network adapter architecture that provides direct PCIe access from multiple CPU sockets to a single NIC, bypassing the inter-processor bus. DPUs in Socket Direct configurations are not tested or validated with DPF.

  • For more details, see Platform Support - Limitations.

• Stale ports after DPU reboot

  • When rebooting DPU, the old DPU service ports won't get deleted from DPU's OVS and would be stale

  • Internal Ref #4174183

  • Workaround: No workaround, known issue, shouldn't affect performance.

• DPU Cluster control-plane connectivity is lost when physical port P0 is down on the worker node

  • Link down of p0 port on the DPU will result in DPU control plane connectivity loss of DPU components.

  • Internal Ref #3751863

  • Workaround: Make sure P0 link is up on the DPU, if down either restart DPU or refer to DOCA troubleshooting https://docs.nvidia.com/networking/display/bfswtroubleshooting. Note: This issue is relevant for Host Trusted deployments only.

• System doesn't recover after DPU Reset

  • When the user triggers a reset of a DPU in any way other than using DPF APIs (e.g. recreation of a DPU CR), the system may not recover.

  • Internal Ref #4521178, #4188044, #4732664

  • Workaround: Power cycle the host. Note that this operation is dangerous and there might be file system corruption which will require triggering the reprovisioning of the DPU. If the system does not recover, reprovision the DPU by deleting the DPU CR (it will be recreated and DPU provisioning will happen). Note: This issue is relevant for Host Trusted deployments where OVN-Kubernetes is used as a primary CNI.

• Leftover CRs if worker is removed from the cluster permanently

  • When a worker was added to a cluster, optionally had DPU provisioned and later was removed from the host cluster permanently, there may be leftover DPF related CRs in both the host cluster and the DPU cluster.

  • Internal Ref #4403130, #4571788

  • Workaround: No workaround, known issue.

• DPUDeployment or DPUSet created in namespace different from the dpf-operator-system namespace do not trigger DPU provisioning

  • When creating a DPUDeployment or a DPUSet in a namespace other than dpf-operator-system, there are no DPU CRs created due to the DPUNode CRs residing in the dpf-operator-system namespace.

  • Internal Ref #4427091

  • Workaround: Create the DPUDeployment or DPUSet in the dpf-operator-system namespace.

• DPUService stops reconciling when DPUCluster is unavailable for long time

  • When the DPUCluster is unavailable for long time (more than 5 mins), changes to DPUServices (also generated ones via DPUDeployment or DPFOperatorConfig) that have happened during that time might not be reflected to the DPUCluster.

  • Internal Ref #4359857

  • Workaround: Recreate DPUServices that are stuck.

• [OVN-Kubernetes DPUService] Lost traffic from workloads to control plane components or Kubernetes services after dpu reboot, port flapping, ovs restart or manual network configuration

  • Connectivity issues between workload pods to control plane components or Kubernetes services may occur after the following events: DPU reboot without host reboot, high speed port flapping (link down/up), ovs restart, DPU network configuration change (for example using "netplan apply" command on DPU). The issues are caused by network configuration that was applied by ovn CNI on DPUs and won't get reapplied automatically.

  • Internal Ref #4188044, #4521178

  • Workaround: Recreate the OVN-Kubernetes node pod on the host to reapply the configuration.

• [OVN-Kubernetes DPUService] No network connectivity for SR-IOV accelerated workload pods after DPU reboot

  • SR-IOV accelerated workload pod is losing its VF interface upon DPU reboot. VF is available on the host however not injected back into the pod.

  • Internal Ref #4236521

  • Workaround: Recreate the SR-IOV accelerated workload pods.

• [Firefly DPUService] Deletion of the Firefly DPUService leaves stale flows in OVS

  • When the Firefly DPUService is deleted after successful deployment or the labels of the serviceDaemonSet are modified, flows are not cleaned up from OVS.

  • Internal Ref #4382535

  • Workaround: Although these flows are very unlikely to cause an issue, reprovisioning the DPU or power cycling the host will bring the OVS in good state. Note that power cycling the host is dangerous and there might be file system corruption which will require triggering the reprovisioning of the DPU.

• [DPUDeployment] Pods of service referenced in spec.serviceChain are restarted when serviceChain changes, causing disruption

  • When updating DPUDeployment.spec.serviceChain with a disruptive upgrade, the affected pods are restarted but DPUDeployment removes the node effect from the corev1.Node in the host cluster proactively. This causes the node to be marked as ready and accept workloads before DPU services are fully operational.

  • Internal Ref: #4686635

  • Workaround: No workaround, known issue.

• Long DPU provisioning time when multiple DPUs are provisioned on the same node

  • In some cases, it may take a long amount of time to provision multiple DPUs on the same node.

  • Internal Ref: #4757191

  • Workaround: If the DPU is stuck in DPU Cluster Config phase after a long installation, please reprovision the DPU and make sure no other DPUs on the same node are being provisioned at the same time

• [DPUDeployment] DPU does not recover to Ready after Node Effect Removal timeout, even when service configuration is reverted

  • When a DPUDeployment is deployed and becomes ready, and then a service is updated with an invalid configuration (e.g. overriding the image with a wrong one in DPUServiceConfiguration), the DPU enters the Node Effect Removal state. If the Node Effect Removal timeout is exceeded, the DPU transitions to Error state. Even after reverting the DPUServiceConfiguration to the previous valid configuration, the DPU does not move back to Ready. The nodeEffectRemovalTimeout in DPFOperatorConfig.spec.provisioningController defaults to 0s (disabled), meaning this issue will not occur with the default configuration. If the timeout is explicitly set to a non-zero value (e.g. 30m), ensure the configured duration allows sufficient time for configuration corrections before the DPU transitions to Error state.

  • Internal Ref: #4952676

  • Workaround: Revert to a working configuration and delete the DPU CR to trigger reprovisioning.

• Leader switch of the provisioning controller pod while the DPU is in the middle of deploying may cause OS installation failure

  • When deploying a DPU without bfbPVCName provided in the DPFOperatorConfig, a leader switch in the DPU provisioning controller (for any reason) during the provisioning process may cause the DPU to transition to the Error phase due to download failures of the BFB or bfb.cfg during OS installing. This issue produces a BFBTransferred condition (FailToInstall reason) in the zero-trust environment and an OSInstalled condition (InstallationTerminated reason) in the host-trusted environment.

  • Internal Ref: #4957820, #4930198

  • Workaround: Delete the DPU CR to trigger reprovisioning.

• DOCA SNAP may fail to reattach a DPUVolumeAttachment that uses VirtioFS or emulated NVMe devices on hot-plugged PFs after a host power cycle

  • After a host power cycle, a DPUVolumeAttachment that uses VirtioFS or emulated NVMe devices on hot-plugged PFs can become not ready and might not be restored automatically.

  • Internal Ref: #4734537

  • Workaround:

    • Step 1: Delete failed DPUVolumeAttachment.

    • Step 2: Create a new DPUVolumeAttachment with the same configuration.

• [Zero Trust] Manual change of DPU mode from DPU to NIC after DPU discovery causes provisioning to stall in Rebooting

  • In a Zero Trust deployment, if a user manually changes the DPU mode from DPU to NIC after DPU discovery, DPU provisioning can remain stuck in the Rebooting phase.

  • Internal Ref: #4984034

  • Workaround: Manually set the DPU mode back to DPU, or delete the DPUDevice CR to trigger DPUDevice recreation.

Last updated: October 14, 2026