Networking NVIDIA UFM Enterprise REST API Guide

PKey GUIDs Rest API

This REST API can be used in UFM Prime environment. 

  • For requests that do not carry a list of GUIDs (for example, creating an empty PKey, or updating QoS), the Consumer broadcasts the same request to every connected provider.

  • For per-PKey GET endpoints, the Consumer broadcasts the GET to every provider and merges the per-provider responses. When a PKey exists on multiple providers the per-provider `guids` lists are concatenated into a single aggregated `guids` list.

  • All PKey GET endpoints accept an optional `site_name` query parameter. When supplied, the Consumer forwards the request only to the named provider and returns that provider's response unchanged (no merging).


Note that the previous PKey GUIDs APIs based on /ufmRest/actions/ are deprecated and being replaced by the new set of APIs below (based on /ufmRest/resources/).

When sharp_enabled and enable_sharp_allocation are enabled in gv.cfg, Creating or updating a PKey will automatically create or update the associated SHARP reservation.

  • Description – allows users to manage PKey GUIDs by getting, adding, and removing GUIDs from PKeys.

  • Request URL – /ufmRest/resources/

  • Main Operations

    • Create an Empty Pkey

    • Add GUIDs to PKey

    • Remove GUIDs from PKey

    • Get a Specific PKey

    • Get all PKeys

    • Set GUIDs for PKey

    • Delete PKey

    • Update QoS for PKey

    • Set SHARP Reservation for Specific Pkey

Create an Empty PKey

  • Description: Allows creating a PKey without GUIDs.

  • Request URL: POST /ufmRest/resources/pkeys/add

  • Request Content Type: Application/json

  • Request Data Example:

    {
     "pkey":"0x12",
     "partition_name":"user_partition",  
     "index0":false,       
     "ip_over_ib":true, 
     "mtu_limit":2,      
     "service_level":0,  
     "rate_limit":2.5     
    }
    


The required parameters are PKey only, all the others are optional and take the default in case they were not sent.

Add GUIDs to PKey

  • Description – adds a list of configured GUIDs to PKey.

  • Request URL – POST /ufmRest/resources/pkeys/

  • Request Content Type – Application/json

  • Request Data Parameters

    Name

    Value

    Default

    Description

    Mandatory/
    Optional

    guids

    ["…",… ".."]
    Each GUID is a hexadecimal string with a minimum length of 16 characters and maximum length of 20 characters

    None

    List of port GUIDs

    Mandatory

    ip_over_ib

    Boolean: true/false

    True

    PKey is a member in a multicast group that uses IP over InfiniBand

    Optional

    index0

    Boolean: true/false

    False

    If true, the API will store the PKey at index 0 of the PKey table of the GUID.

    Optional

    membership

    “full”, “limited”

    "full"

    • “full”- members with full membership can communicate with all hosts (members) within the network/partition

    • “limited” - members with limited membership cannot communicate with other members with limited membership. However, communication is allowed between every other combination of membership types.

    Optional

    memberships

    [“full”, “limited”, ...]

    List of “full” or “limited” comma-separated strings. It must be the same length as the GUIDs list. Each value by an index represents a GUID membership.

    []

    List of memberships to allow users to create or modify different membership per GUID in same PKey. GUID index 0 in the “guids” list will take index 0 from the memberships list and so on (by order)

    This parameter is optional. This parameter conflicts with the “membership” parameter. Users must select either a list of memberships or just one membership for all GUIDs.

    pkey

    Hexadecimal string between "0x0"-"0x7fff" exclusive

    None

    Network PKey

    Mandatory

    partition_name

    String with at least 1 character and a  maximum of 40 characters

    api_pkey_<pkey>

    Network Pkey Name

    Optional


  • Response – N/A

  • Status Codes

    • 200 – OK

    • 400 – bad request

  • Request Data Example

    {
    "guids": ["0002c903000e0b72", "0002c903000e0b73"],
    "ip_over_ib": false,
    "index0": true,        
    "membership": ["full", "full"],
    "partition_name":"ufm_partition",  
    "pkey": "0x0a12"
    }
    


  • Response – N/A

  • Status Codes

    • 200 – OK

    • 400 – bad request

Remove GUIDs from PKey

  • Description – remove a list of configured GUIDs from PKey

  • Request URL – DELETE /ufmRest/resources/pkeys/<pkey>/guids/<guid1>,<guid2>,…

  • Request Content Type – Application/json

  • Request Data Parameters

    Name

    Value

    Default

    Description

    Mandatory/
    Optional

    guids

    ["…",… ".."]
    Each GUID is a hexadecimal string with a minimum length of 16 characters and maximum length of 20 characters

    None

    List of port GUIDs

    Mandatory

    pkey

    Hexadecimal string between "0x0"-"0x7fff" exclusive

    None

    Network PKey

    Mandatory


  • Status Codes

    • 200 – OK

    • 400 Bad request

Get Specific PKey

  • Description – returns information about the specified PKey. The "guids_data" parameter enables retrieving information on the GUIDs of the specified PKey as well.

  • Request URL – GET /ufmRest/resources/pkeys/<pkey>?guids_data=<boolean>

    • <pkey> – hexadecimal pkey in the range of 0x0-0x7fff

  • Request Content Type – Application/json

  • Response Data Example – without GUID data

    {
        "partition": "api_pkey_0x1", 
        "ip_over_ib": true
    }
    


  • Response Data Example – with GUID data and sharp_state

    {
    "guids":[
    {
    "guid": "98039b0300860ace",
    "membership": "full",
    "index0": false
    },
    {
    "guid": "98039b0300860acf",
    "membership": "full",
    "index0": false
    }
    ],
    "partition": "api_pkey_0x1",
    "ip_over_ib": true,
    "sharp_enabled": true
    }
    


  • Status Codes

    • 200 – OK

Get All PKeys

  • Description – returns a list of all PKeys (with or without their associated GUIDs data).

  • Request URL – GET /ufmRest/resources/pkeys?guids_data=<boolean>&qos_conf=<boolean>

    • guids_data flag

      • False – returns a list of all PKeys

      • True – returns a list of all PKeys including their associated GUIDs

    • qos_conf flag

      • False – returns a list of all PKeys without QoS configuration

      • True – returns a list of all PKeys including their associated QoS configuration

    • port_info flag

      • False – returns a list of all PKeys without port information details

      • True – returns a list of all PKeys including their associated port information

    • max_ports flag

      • <positive-int-value> – returns a list of all PKeys with port information details for ports with number less than or equal to the provided value

    • sharp_state flag

      • True – Include the sharp_enabled key for each PKey to reflect its SHARP enablement state.

  • Request Content Type

  • Application/json

  • Response Data Example – without GUID data

    ["0x1", "0x2", "0x3"]
    


  • Response Data Example – with GUID data

    {
    "0x3": {
    "guids": [
    {
    "membership": null, "guid": "0002c903000e0b74",
    "index0": false
    },
    {
    "membership": null, "guid": "0002c903000e0b75",
    "index0": false
    }
    ],
    "ip_over_ib": true, "partition": "api_pkey_0x3",
    "qos_conf":{"mtu_limit":4,
                 "Rate_limit":300,
                  "Service_level":3}
    }, "0x2": {
    "guids": [
    {
    "membership": null, "guid": "0002c903000e0b76",
    "index0": false
    },
    {
    "membership": null, "guid": "0002c903000e0b77",
    "index0": false
    }
    ],
    "ip_over_ib": true, "partition": "api_pkey_0x2",
    "qos_conf":{"mtu_limit":4,
                 "Rate_limit":300,
                  "Service_level":3
    }, "0x1": {
    "guids": [
    {
    "membership": "full", "guid": "0002c903000e0b78",
    "index0": false
    }
    ],
    "ip_over_ib": true, "partition": "api_pkey_0x1",
    "qos_conf":{"mtu_limit":4,
                 "Rate_limit":300,
                  "Service_level":3
    }
    }
    


  • Status Codes

    • 200 – OK

  • Response Data Example – with sharp_state

    {
    "0x7fff":{
    "sharp_enabled": false
    },
    "0x1":{
    "sharp_enabled": true
    },
    "0x2":{
    "sharp_enabled": true
    }
    }
    


  • Status Codes

    • 200 – OK

Set/Update PKey GUIDs

  • Description – sets/updates a list of configured GUIDs for PKey (or overwrites the current list, if found), including Quality-of-Service (QoS) parameters.

  • Request URL – PUT /ufmRest/resources/pkeys/

  • Request Content Type – Application/json

  • Request Data Parameters

    Name

    Value

    Default

    Description

    Mandatory/
    Optional

    guids

    ["…",… ".."]
    Each GUID is a hexadecimal string with a minimum length of 16 characters and maximum length of 20 characters

    None

    List of port GUIDs

    Mandatory

    ip_over_ib

    Boolean: true/false

    True

    PKey is a member in a multicast group that uses IP over InfiniBand

    Optional

    index0

    Boolean: true/false

    False

    If true, the API will store the PKey at index 0 of the PKey table of the GUID.

    Optional

    membership

    “full”, “limited”

    "full"

    • “full”- members with full membership can communicate with all hosts (members) within the network/partition

    • “limited” - members with limited membership cannot communicate with other members with limited membership. However, communication is allowed between every other combination of membership types.

    Optional

    memberships

    [“full”, “limited”, ...]

    List of “full” or “limited” comma-separated strings. It must be the same length as the GUIDs list. Each value by an index represents a GUID membership.

    []

    List of memberships to allow users to create or modify different membership per GUID in same PKey. GUID index 0 in the “guids” list will take index 0 from the memberships list and so on (by order)

    This parameter is optional. This parameter conflicts with the “membership” parameter. Users must select either a list of memberships or just one membership for all GUIDs.

    mtu_limit 

    value can be 2k or 4k 

    MTU limit 

    Optional 

    service_level 

    value can be range from 0-15 

    Service level 

    Optional 

    rate_limit 

    value can be one of the following: 2.5, 10, 30, 5, 20, 40, 60, 80, 120, 14, 56, 112, 168, 25, 100, 200, or 300 

    2.5 

    Rate Limit 

    Optional 

    pkey


    Hexadecimal string between "0x0"-"0x7fff" exclusive

    None


    Network PKey


    Mandatory


    partition_name

    String with at least 1 character and a  maximum of 40 characters

    api_pkey_<pkey>

    Network Pkey Name

    Optional


  • Request Data Example

    { 
      "guids": ["0002c903000e0b72", "0002c903000e0b73"], 
      "ip_over_ib" : false, "index0": true, 
      "index0": true, 
      "membership": ["full", "full"],
      "mtu_limit":2, 
      "service_level":0, 
      "rate_limit":2.5,
      "partition_name":"ufm_partition",      
      "pkey": "0x0a12"
    }
    


  • Response – N/A

  • Status Codes

    • 200 – OK

    • 400 – bad request

    To use this API to trigger Mellanox Scalable Hierarchical Aggregation and Reduction Protocol (SHARP)™ allocations and deallocations, please refer to "NVIDIA SHARP REST API".


Add Hosts to PKey

  • Description - Allows the user to create new Partition key (Pkey) assignments, including all the ports allocated to sent host names. UFM-SLURM plugin reads the following API attributes (ip_over_ib, index0, membership) from ufm_slurm.conf file, and sends them in request body. 

  • Request URL - POST /ufmRest/resources/pkeys/hosts 

  • Status Codes

    • 200 – OK

    • 400 – not found

  • Request Data Example: 
    The response is a job, and its status depends on the action output. 

    {   
      "hosts_names":"r-ufm51,r-ufm77", 
      "ip_over_ib": true, 
      "index0": false, 
      "membership": "full", 
      "pkey": "0xa12" 
    }
    


Remove Hosts from PKey

  • Description - Using this API allowed the user to remove a list of configured Hosts GUIDs from PKey. 

  • Request URL - DELETE /ufmRest/resources/pkeys/<pkey>/hosts/<host_name1>,<host_name2>,… 

  • Status Codes

    • 200 – OK

    • 400 – not found

Delete PKey

  • Description – deletes a PKey and all of its configured GUIDs.

  • Request URL – DELETE /ufmRest/resources/pkeys/<pkey>

  • Status Codes

    • 200 – OK

    • 404 – not found

Update PKey QoS

  • Description – updates the QoS configuration for a specific PKey.

  • Request URL – PUT /ufmRest/resources/pkeys/qos_conf

  • Request Data Example

    {
      "mtu_limit": 4,
      "service_level": 5,
      "rate_limit": 2,
      "pkey": "0x9"
    }
    


  • Notes

    • mtu_limit value can be 2k or 4k

    • service_lever value can be range from 0-15

    • rate_limit value can be one of the following: 2.5, 10, 30, 5, 20, 40, 60, 80, 120, 14, 56, 112, 168, 25, 100, 200, or 300

    • Restarting UFM is required for the PKey QoS configuration to take effect

  • Status Codes

    • 200 – OK
      400 – bad request

PKey Version (Last Updated)

  • Description – returns the time when PKey data was last modified.

  • Request URL – GET /ufmRest/resources/pkeys/last_updated

  • Request Data Example

    {
        "last_updated": "Thu Sep  3 11:42:39 UTC 2020"
    }
    


  • Notes

    • Last updated data is not persistent and will be reset when UFM is restarted

    • By default, when no updates are done on PKey data, last_updated value will be null

    • The value of last_updated returned by this REST API will be updated when one of the following REST APIs are called:

      • Add GUIDs to PKey

      • Remove GUIDs from PKey

      • Set GUIDs for PKey

      • Delete PKey

      • Update QoS for PKey

Set SHARP Reservation for Pkey

  • Description: Allows enabling/disabling the sharp reservation for the pkey.

  • Request URL: PUT /ufmRest/resources/pkeys/<pkey_id>/sharp-reservation

  • Request Content Type: Application/json

  • Request Data ParametersNameValueDefaultDescriptionMandatory/Optionalaction“enable”, “disable”None“enable”- enable the sharp reservation for the given Pkey.“disable” - disable the sharp reservation for the given Pkey.Mandatory

  • Request Data Example:

    {"action": "enable"}
    


Note: Ensure that both sharp_enabled and enable_sharp_allocation are set to true in gv.cfg before using this API.

Last updated: