Skip to content

Models#

An agent's model call is the one request it always makes. Routing it through the platform is what turns spending limits, PII masking, and per-call audit from aspirations into enforcement.

Managed inference#

The platform holds the provider credential and proxies every call. The agent points its SDK at the platform instead of at the provider, and never holds a model API key.

graph LR
    agent[Agent in sandbox] -->|no key| proxy[LLM proxy]
    proxy --> limits{Spending limit}
    proxy --> mask[PII masking]
    proxy --> audit[(Audit trail)]
    proxy --> provider[Bedrock · Foundry]

Enabling it takes two steps: the install configures a provider, and a policy opts in.

name: agent-base
managedInference:
  enabled: true
  provider: bedrock

Omit managedInference and sandboxes under that policy have no model access at all. It is opt-in, not opt-out.

Provider routes#

Agents call the platform on provider-shaped paths, so an unmodified SDK works by changing only its base URL.

All paths below are relative to /v1/projects/{projectId}. The project-scoped form works for every caller, and it is the form a sandbox is given.

Provider Route
Bedrock /llm/bedrock/{region}/{path}
Foundry — Claude /llm/azure/anthropic/{path}
Foundry — GPT /llm/azure/openai/{path}
Bedrock Mantle — Claude /llm/bedrock-mantle/anthropic/{region}/{path}
Bedrock Mantle — GPT /llm/bedrock-mantle/openai/{region}/{path}
OpenAI /llm/openai/{path}
OpenAI — realtime voice /llm/openai/v1/realtime (WebSocket)

A sandbox never builds these paths itself. The platform gives it the project-scoped base URL its SDK reads, so changing that one variable is all an agent needs. See What the sandbox receives.

Ask a running install what it has configured:

curl -fsS -H "Authorization: Bearer $LENS_AGENTS_TOKEN" \
  https://agents.example.com/v1/inference/providers

A policy may only name a provider in that list. Anything else has no route behind it.

Provider selection has to agree#

Three places name a provider, and they must match or the agent will not answer:

  1. The install — which providers were configured. See Inference providers.
  2. The policy — managedInference.provider.
  3. The agent — whatever its own configuration or environment tells it to use.

A mismatch is the most common reason a healthy-looking agent stays silent. nexusctl sandbox describe shows the effective policy, which is the fastest way to check the middle one.

What proxying buys you#

Metering. Every call is priced and attributed to the org, user, agent, and sandbox that made it. Spending limits are enforced at the proxy — on breach it returns 429 and the sandbox keeps running, losing model access rather than its process and disk.

Masking. PII can be replaced before the prompt leaves your network and restored in the response. Masking is fail-closed by default: if masking fails, the request does not go out. See Privacy and PII controls.

Audit. Actor, project, provider, model, token counts, cost, result, and duration for every call. See Audit trail.

Cost visibility. Usage-cost summaries and timeseries per org, sliceable by actor.

None of this is available for a call the platform never sees.

Agents that bring their own model#

An agent can also hold its own provider key and reach the provider through an allowed domain instead:

allowedDomains:
  - pattern: api.anthropic.com
    verdict: allow
    transport: upstream

The call is still policy-checked and audited at the network level. What you lose is per-call metering, spending limits, and PII masking, because the platform is forwarding bytes rather than understanding a model request.

Prefer managed inference wherever the choice exists.

Additional model providers are configured to customer requirements. Talk to us.