Cost Optimization
Reducing AI cost starts with knowing where it goes, then changing the model, the prompt or the traffic. ANTS Platform helps with each step.
ANTS Platform does not change your application. It measures, tests and recommends; you apply the change in your code or in Budget AI routing.
1. Find where the money goes
Use Gauge AI → Cost Dashboard to find the models, trace names (agents) and users with the highest spend, and Gauge AI → Cost Performance Metrics to see whether that spend buys better quality scores. See Cost & Latency Dashboards and Cost & Performance Metrics.
2. Test a cheaper model
Run the AI Optimizer on a costly agent. It replays the agent's real traces on up to three cheaper models with your own provider keys, scores every output with LLM-as-a-Judge, and recommends a model only if its average score is at least the baseline's. Use batch inference for large samples to lower the provider cost of the test.
3. Shorten prompts
Long system prompts and context are billed on every call. Compare Units by type (input versus output tokens) in Cost Performance Metrics → Model Usage, and open expensive traces in Discover AI → Tracing to see what is being sent.
4. Cache and route through Budget AI
If your traffic goes through Budget AI, you can cache responses for a model group, route to cheaper deployments, fall back between providers, and cap spend with budgets. See Model Groups & Routing and Budgets & Limits.
5. Keep prices accurate
Savings estimates and dashboards are only as good as the model prices. Add definitions for custom or fine-tuned models and override managed prices with your negotiated rates.
6. Sample ingestion volume
Ingested traces, observations and scores count toward credits on credit-based plans. If you do not need every trace, the Python SDK can sample a share of traces:
You can also set ANTS_PLATFORM_SAMPLE_RATE=0.2. Sampled-out traces are not sent and do not appear in dashboards, so keep the full rate for traffic you need to audit. See Credits.