Inference providers#
Agents need a model. Lens Agents can proxy every LLM call an agent makes, which is what makes spending limits, PII masking, and per-call audit possible. That is managed inference: the platform holds the provider credential, the agent never sees it, and each call passes through the platform's LLM proxy.
Managed inference is opt-in per policy. Configuring a provider here makes it available; a policy has to select it before any sandbox can use it.
Supported providers#
| Provider key | Backend | Model families |
|---|---|---|
bedrock |
Amazon Bedrock | Claude |
azure |
Microsoft Foundry | Claude (Anthropic Messages API) and GPT (OpenAI Chat Completions and Responses APIs) |
bedrock-mantle |
Amazon Bedrock Mantle | Claude and GPT off the same Bedrock key |
Ask which providers a running install has configured:
curl -fsS -H "Authorization: Bearer $LENS_AGENTS_TOKEN" \
https://agents.example.com/v1/inference/providers
{ "providers": ["bedrock", "bedrock-mantle"] }
A policy may only select a provider in that list. Selecting anything else has no proxy route behind it and returns 404.
AWS Bedrock#
Bedrock is the default provider. It needs credentials, resolved one of two ways.
On a cluster with an IAM role#
On EKS, or any cluster where the platform's service account can assume a role, attach the role and leave inference.bedrock empty. The AWS SDK default provider chain resolves it.
helm upgrade --install lens-agents oci://ghcr.io/lensapp/lens-agents \
--set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=\
"arn:aws:iam::123456789012:role/lens-agents-bedrock" \
...
This is the preferred shape: no long-lived key is stored in the cluster at all.
With a Bedrock API key#
Where there is no IRSA or instance role — local clusters, on-prem clusters outside AWS — supply an Amazon Bedrock API key. The platform forwards it upstream as a bearer token.
kubectl create secret generic lens-agents-bedrock \
--from-literal=NEXUS_BEDROCK_TOKEN="<bedrock-api-key>"
helm upgrade --install lens-agents oci://ghcr.io/lensapp/lens-agents \
--set inference.bedrock.existingSecret=lens-agents-bedrock \
...
Setting a Bedrock token also exposes Bedrock Mantle as a selectable provider. Mantle is AWS's Anthropic- and OpenAI-compatible endpoint, serving both Claude and GPT off the same key. It needs no extra configuration — the host is derived from the request region.
Microsoft Foundry#
One Foundry resource and key serve both model families. A policy selecting azure is authorized for both the Claude and GPT surfaces.
kubectl create secret generic lens-agents-azure \
--from-literal=NEXUS_AZURE_TOKEN="<azure-api-key>"
helm upgrade --install lens-agents oci://ghcr.io/lensapp/lens-agents \
--set inference.azure.baseUrl="https://<resource>.services.ai.azure.com" \
--set inference.azure.existingSecret=lens-agents-azure \
...
Use the Foundry resource root, not the Azure OpenAI host
inference.azure.baseUrl is the resource root on the *.services.ai.azure.com host — not the Azure OpenAI *.openai.azure.com host. Leave the surface suffix off: the proxy appends /anthropic for Claude and /openai for GPT itself. Copy the host from a deployment's Target URI in the Foundry portal; the key is that deployment's Key.
inference.azure.anthropic.model sets the default Claude deployment name for the /anthropic surface. GPT takes its model per request, so there is no openai counterpart.
Values#
| Value | Default | Purpose |
|---|---|---|
inference.bedrock.token |
(empty) | Bedrock API key, forwarded as a bearer token. Empty falls back to the AWS SDK default chain. |
inference.bedrock.existingSecret |
(empty) | Read the token from an existing Secret instead. |
inference.bedrock.existingSecretKey |
NEXUS_BEDROCK_TOKEN |
Key within that Secret. |
inference.azure.baseUrl |
(empty) | Foundry resource root. Empty disables Azure. |
inference.azure.token |
(empty) | Azure API key, forwarded as the api-key header. |
inference.azure.existingSecret |
(empty) | Read the token from an existing Secret instead. |
inference.azure.existingSecretKey |
NEXUS_AZURE_TOKEN |
Key within that Secret. |
inference.azure.anthropic.model |
(empty) | Default Claude deployment name. |
Prefer existingSecret over token in every environment. A --set value lands in your shell history and in the Helm release.
Turning it on for agents#
Configuring a provider is half the job. A policy has to enable managed inference and name the provider:
name: agent-base
managedInference:
enabled: true
provider: bedrock
Sandboxes governed by that policy reach the model through the platform. Sandboxes without it have no model access at all.
Three places must agree, or the agent will not answer: the provider configured on the install, the provider named in the policy, and whatever provider the agent image itself is told to use through its environment.
What you get from proxying#
Because every call passes through the platform:
- Spending is metered per call and enforced against spending limits by org, user, agent, or sandbox.
- PII can be masked before the prompt leaves your network, and unmasked in the response. See Privacy and PII controls.
- Every call is audited with the actor, project, and cost. See Audit trail.
Additional providers are configured to customer requirements. Talk to us.
Related#
- Models — how agents select models at runtime
- Spending limits — caps by org, user, agent, and sandbox
- Policies — enabling managed inference for a set of sandboxes