Networking Minerva User Guide
Networking Minerva User Guide
Breadcrumbs

Configure an On-Premises LLM

Configure the NVIDIA Minerva GenAI backend during initial onboarding when the deployment must use a customer-hosted, on-premises large language model (LLM) instead of a preconfigured NVIDIA internal gateway.

The LLM provider is configured centrally in NVIDIA Minerva. Client applications do not supply an LLM key when they connect. For general instructions on using the natural-language assistant after configuration, see Ask Sherlock.

Before You Begin

  • Confirm that the on-premises inference service exposes an OpenAI-compatible API that supports tool calling.

  • Obtain the inference URL, exact model identifier, and API credential expected by the service.

  • Confirm that NVIDIA Minerva services can resolve and reach the inference URL.

  • If the service uses a private certificate authority, install the required CA trust for the NVIDIA Minerva services before testing.

  • Confirm the selected model is approved and sized for the expected request volume and context length.

Configure the Provider

  1. In the NVIDIA Minerva UI, open Settings.

  2. Go to Configs → default → config → api → genai → global.

  3. Set enable to true.

  4. Under openaiTools, configure:

    • Inference URL: <https://your.inference-api.com>

    • Model: <claude-sonnet-4-6>

    • API key: a valid Inference API key

Sherlock is tested against Claude Sonnet 4.6. This feature is in beta.

The LLM key is stored and used by NVIDIA Minerva.

Validate Onboarding

  1. Open Ask Sherlock and start a new conversation.

  2. Submit a simple cluster question that requires live data, such as: Summarize active incidents.

  3. Confirm that the assistant returns a grounded response rather than a provider, authentication, or tool-calling error.

  4. Submit a second question that requires a tool call, such as: Show interface errors from the last hour.

  5. Review the NVIDIA Minerva service logs for TLS, authentication, model-not-found, timeout, or tool-calling errors.

  6. Record the validated inference URL and model identifier in the deployment handoff. Store the credential only in the approved secret-management location.

If basic chat succeeds but data questions fail, verify that the selected model supports OpenAI-style tool calling. If all requests fail, verify network routing, DNS, TLS trust, credentials, the model identifier, and the inference URL path.

Last updated: