Docs/Budget Ai/Providers And Credentials

Providers & Credentials

A provider credential is the account Budget AI uses to call a provider on your behalf: a provider family, an optional base URL, and a secret. Credentials belong to a project and are managed in Budget AI → Models & Routing → Provider credentials. Applications never see them; they hold only Budget AI keys.

Supported provider families

Every family accepts the same OpenAI-style request from your application. Budget AI translates it for the provider and translates the answer back, including streamed responses.

Family	Base URL	Also needed	Secret	Embeddings
OpenAI-compatible	Optional. Default `https://api.openai.com/v1`. Set it for any other OpenAI-compatible API.	None	API key, sent as `Authorization: Bearer`	Yes
Azure OpenAI	Required: your resource endpoint	API version (optional, default `2025-02-01-preview`)	API key, sent as `api-key`	No
Anthropic	Optional. Default `https://api.anthropic.com`	None	API key, sent as `x-api-key`	No
AWS Bedrock	Optional. Default `https://bedrock-runtime.<region>.amazonaws.com`	Region (required)	Access key JSON: `{"accessKeyId": "...", "secretAccessKey": "..."}`	No
Google Vertex AI	Optional. Default is the regional Vertex AI endpoint, or the Gemini API when you use a Gemini API key	Region (required)	Service account JSON, or a Gemini API key	No

Notes per family:

OpenAI-compatible requests go to {base URL}/chat/completions and {base URL}/embeddings. Streamed requests ask the provider for a final usage chunk, so streamed calls are costed from real usage.
Azure OpenAI requests go to {base URL}/openai/deployments/{provider model}/chat/completions?api-version=.... Use the Azure deployment name as the provider model. A base URL that already ends in /openai/deployments is not repeated.
Anthropic requests are translated to the Messages API. When the caller sets no max_tokens, Budget AI sends 4096.
AWS Bedrock requests use the Bedrock Converse API (and its streaming variant), signed with AWS Signature Version 4.
Google Vertex AI requests use generateContent and streamGenerateContent. With a service account, Budget AI obtains and refreshes the access token itself.

A model group can mix families. For embeddings, models whose family has no embeddings API are skipped, so a mixed group still serves embeddings from its OpenAI-compatible models. See Model Groups & Routing.

Add a credential

Open Provider credentials

Go to Budget AI → Models & Routing and select Add credential under Provider credentials. You need gatewayAi.models:admin.

Fill in the form

Enter a Name (unique in the project), choose the Provider family, and fill in Base URL, API version or Region where shown. Paste the Secret.

Confirm with step-up verification

Creating or changing a credential asks for step-up verification.

The secret is never shown again. The credentials table shows the last four characters, when it was last rotated, where it is stored, and how many models use it.

Import an existing LLM Connection

If the project already has an LLM Connection for OpenAI, Azure OpenAI or Anthropic, select Import LLM Connection, choose the connection and give the credential a name. Budget AI copies the connection's encrypted secret, base URL and family without decrypting it. Imported credentials are marked imported.

Base URL rules

A base URL receives the credential's secret, so Budget AI refuses one that:

is not a valid http:// or https:// URL;
embeds a username or password;
points at a loopback, private, carrier-grade NAT or link-local address, localhost, or a cloud metadata host.

Trailing slashes are ignored. Leave the field empty to use the family's default.

Where provider secrets are stored

Each credential's secret is stored in one of two places. The Stored in column of the credentials table shows which.

Store	How it works
Database	The secret is encrypted with AES-256-GCM under the platform's `ENCRYPTION_KEY` and stored in the platform database. This is the default.
AWS Secrets Manager	The secret is written to AWS Secrets Manager at `<prefix>/<project id>/<credential id>` and the database holds only that reference. The path is derived by the platform and is never accepted as input.

The store for new credentials is an instance setting (GATEWAY_SECRETS_STORE=aws, see Self-Hosting & Configuration). Project Settings → Budget AI → Provider secret storage shows which store is in use, whether the managed store is reachable, and how many credentials are still in the database.

When the managed store is configured, credentials still in the database show a Move action. Moving writes the secret to AWS Secrets Manager, reads it back to confirm, and only then switches the credential to the new reference. Step-up verification is required.

ℹ

Gateway replicas keep provider secrets in memory and refresh secrets held in AWS Secrets Manager every five minutes by default. If a secret cannot be read, a replica keeps the copy it already had. A credential with no readable secret is left out of routing until it can be read.

Rotate a provider secret

Budget AI cannot create or revoke keys at the provider. Rotation is a safe swap with a check in the middle:

Create a replacement key in the provider's console

Paste it into Budget AI

Select Rotate on the credential and paste the New provider secret. Budget AI stores it, reads it back, and switches every gateway replica to it. Traffic keeps flowing while it propagates. Step-up verification is required.

Wait for traffic to be served with it

The rotate panel shows Since the last rotation: how many of the credential's models have served or failed a request since the swap.

Revoke the old key in the provider's console

The credentials table shows when each secret was last rotated, or never rotated.

Provider spend caps

A leaked provider secret is bounded only by a spend cap set at the provider. The credentials table marks credentials with no cap recorded as a reminder to set one in the provider's console.

Change or delete a credential

Edit changes the name, base URL, API version or region, and optionally the secret (leave New secret empty to keep it). Step-up verification is required.
Delete is refused while any model uses the credential. Remove or re-point those models first. When a credential held in AWS Secrets Manager is deleted, its managed secret is scheduled for deletion with a recovery window.

Every create, change, rotation, move, import and delete is written to the audit log.

Health checks

Health checks are off by default. Turn them on per project in Project Settings → Budget AI → Health checks, then choose a mode and How often (every minute, 5 minutes, 15 minutes or hour).

Mode	What it does	Cost
Check the credential only	Asks the provider to list models with the credential. Catches a dead or revoked key, not a withdrawn model.	Free
Send a one-token request	Sends a one-token completion to each exact model. Catches a dead key and a dead model.	Billable for every model on every check, whether or not anyone is using it

Health checks run inside the gateway, cover every enabled model in the project, use no Budget AI key, create no trace and never count against limits or budgets. The result shows as each model's status in Models & Routing and on the Model health card of the Overview. A model's status is the newer of its last health check and its last real request.

Health checks report status only. Taking a failing model out of rotation is done by benching, which is driven by real traffic. See Model Groups & Routing.