Getting Started
This guide takes a project from nothing to a first traced request through Budget AI.
Before you start
- Budget AI is available to your organization and to you. See What you need to turn it on.
- You hold Budget AI admin access in the project, for example as Owner or Department Admin.
- You have an API key or credentials for at least one provider: OpenAI or an OpenAI-compatible API, Azure OpenAI, Anthropic, AWS Bedrock or Google Vertex AI.
- The model you plan to use has a price in your project's model prices, so requests are costed. Hard budgets refuse models with no price.
Turn on serving for the project
Open Project Settings → Budget AI and turn on Serve gateway traffic for this project.
The Budget AI address section on the same tab shows the base URL to give an OpenAI-compatible SDK. It already ends in /v1. If it reads "The gateway address is not configured on this instance", the deployment has no public gateway address yet; see Self-Hosting & Configuration.
While you are here, check Key policy and project limits: the default and maximum key expiry (90 and 365 days unless changed) and optional project-wide requests and tokens per minute.
Add a provider credential
Open Budget AI → Models & Routing. Under Provider credentials, select Add credential, then enter:
- Name: a label such as
openai-prod. - Provider family: OpenAI-compatible, Azure OpenAI, Anthropic, AWS Bedrock or Google Vertex AI.
- Base URL, API version or Region where the family needs one.
- Secret: the provider API key, or the JSON the family expects.
You are asked for step-up verification before the credential is saved. If the project already has an LLM Connection for the provider, Import LLM Connection reuses it without re-entering the secret. Details per family are in Providers & Credentials.
Create a model group
Under Model groups, select New model group.
- Name applications call: the value clients send as
model, for exampleprod-chat. Letters, digits and. _ : / -, starting with a letter or digit. - Models: add at least one. Each model is a deployment: a credential, a provider model (for example
gpt-4o-mini) and a weight.
A group with no models refuses every call. You can add more models, change the routing strategy and set fallbacks later. See Model Groups & Routing.
Issue a key
Open Budget AI → Keys and select Issue key.
- Choose Show me the key to see it yourself, or Send a collection link to let a developer collect it.
- Enter an Alias such as
checkout-service. - Select the Model groups the key may call.
- Set Expires in (days). Every key expires.
- Optionally set a Budget in USD, and under Advanced settings the rate limits, recording mode, org unit, tags and allowed source networks.
Complete step-up verification. The key is shown once, in the form sk-ants- followed by 40 characters, with ready-to-copy snippets. Store it in your secret manager; it cannot be shown again. See Keys.
Send a request
Replace the address, key and model group with your own.
If the key is granted exactly one model group, model can be left out and the request goes to that group.
Find the trace
The response carries an x-ants-request-id header. That value is the trace id. Open Tracing, set the Source filter to Gateway, and open the trace to see input, output, cost, latency and the Gateway panel with key, model group, deployment and attempts. The trace is named after the model group.
The request also appears on Budget AI → Overview and against the key on Budget AI → Keys.
Common first-request errors
| Response | Meaning | Fix |
|---|---|---|
401 invalid_api_key | The key is missing, mistyped or deleted, or serving is off for the project | Send Authorization: Bearer <key> with the full key, and check Serve gateway traffic for this project |
403 model_not_allowed | The key is not granted that model group, the group does not exist, or model was left out on a key with several groups | Check the group name, or grant the key the group |
503 no_deployment_available | The group has no enabled model with weight above 0 | Add or resume a model in the group |
503 gateway_config_stale | The gateway cannot confirm its configuration is current | Retry after a few seconds; if it persists, see Self-Hosting & Configuration |
409 budget_unpriced_model | A hard budget applies and the provider model has no price | Add a model price |
A newly created key, model group or credential reaches every gateway replica within about a second. If the save message reads "Saved, but not yet in force", the change applies within 60 seconds.
All codes are listed in the API Reference.