LLM Connections
An LLM connection stores the credentials ANTS Platform uses to call a model provider on your behalf. Connections are per project. Your provider bills you for these calls.
These features use LLM connections:
- Playground
- Prompt experiments
- LLM-as-a-Judge evaluators and the default evaluation model
- AI Optimizer, including batch inference on Bedrock and Vertex AI
Trace ingestion does not use LLM connections. SDKs send data with your project API keys. Traffic through Budget AI uses the provider credentials configured in Budget AI, not these connections.
Manage connections in Project Settings > LLM Connections. Among the built-in roles, only Owner can view and manage them today. See Members & Roles.
Supported adapters
| LLM adapter | Credentials | Notes |
|---|---|---|
openai | API key | Optional API base URL for OpenAI-compatible endpoints. Optional extra headers. |
anthropic | API key | Optional API base URL. |
azure | API key | API base URL is required, in the form https://<instance>.openai.azure.com/openai/deployments. Use the Azure deployment name as the model name. |
bedrock | AWS region, plus access key ID and secret access key | On self-hosted deployments the AWS keys are optional. Without them the AWS SDK default credential chain is used, such as environment variables or an IAM role. Model names are Bedrock inference profile IDs. |
google-vertex-ai | Service account JSON key | The service account needs the Vertex AI User role. The Google Cloud project comes from the key. Location is optional and defaults to us-central1. |
google-ai-studio | API key |
Add a connection
Open the form
In Project Settings > LLM Connections, select Add LLM Connection.
Choose the adapter and name it
Pick the LLM adapter. Provider name identifies the connection inside ANTS Platform, for example openai-prod. It is the provider name you see in model pickers.
Enter credentials
Fill in the credentials for the adapter. For Bedrock and Vertex AI, check I intend to use Batch Inference for LLM optimizer if you want the AI Optimizer to use batch jobs. See Bedrock setup and Vertex AI setup.
Choose models
Enable default models makes the adapter's built-in model list available. Add Custom models for any other model names the endpoint accepts, such as fine-tuned models, Azure deployment names or Bedrock inference profile IDs.
Advanced settings
Show advanced settings reveals the API Base URL (leave blank for the provider default), the Vertex AI Location, and Extra Headers for the openai adapter.
Save
Select Create connection. ANTS Platform first sends a test request with the connection and only saves it if the test succeeds. You may be asked to verify your identity before the connection is saved.
Security
- API keys, service account keys and extra header values are stored encrypted. After saving, only a masked key is shown.
- Editing a connection runs the test again before the change is saved.
- You can also list and create or update connections with the public API:
GET /api/public/llm-connectionsandPUT /api/public/llm-connections. Secrets are never returned. See API Reference.