Docs/Llmops/Remote Experiments

Remote Experiments

A remote experiment lets your team start an experiment that runs in your own service, from the dataset page in ANTS Platform. You set a webhook URL and a default config on the dataset once. After that, anyone with access selects Run, optionally edits the config, and ANTS Platform calls your service. Your service runs the experiment and writes the results back as a dataset run.

Use it when your experiment needs your own code, data or infrastructure, but you want people to start it without running a script.

Set up the trigger

Open the dialog

Open the dataset and select New experiment. In the Custom Experiment card, select the settings button titled Set up remote experiment in UI trigger.

Enter the URL and default config

URL: the HTTPS endpoint of your service, for example https://experiments.example.com/run.
Default config: a JSON object that is sent with every run, for example the model or prompt version to test. People can change it before each run.

Save

Select Set up. The Custom Experiment card now shows Run, plus a button to edit or delete the trigger.

Among the built-in roles, only Owner can set up and run remote experiments today. See Members & Roles.

Run it

Select New experiment > Custom Experiment > Run. The Run remote experiment dialog shows the config. Edit it if needed and select Run. ANTS Platform shows Experiment started when your service accepts the request, or Failed to start experiment otherwise.

The request your service receives

ANTS Platform sends one POST request from its servers to your URL:

http

POST /run HTTP/1.1 Content-Type: application/json { "projectId": "<project id>", "datasetId": "<dataset id>", "datasetName": "support/regression", "payload": "{\"model\": \"gpt-4o-mini\", \"promptVersion\": 3}" }

Field	Meaning
`projectId`	The ANTS Platform project.
`datasetId`	The dataset ID.
`datasetName`	The dataset name. Use it with `get_dataset` in the SDK.
`payload`	The config from the dialog, as a JSON-encoded string. Parse it before use.

Rules:

Respond with any 2xx status within 10 seconds. Any other status, a timeout or a network error counts as a failed start.
Start the experiment in the background and respond right away. The request does not wait for the experiment to finish.
The request carries no signature or credentials. Protect your endpoint yourself, for example with a hard-to-guess path, network rules, or by checking projectId and datasetId against values you expect.
Your endpoint must be reachable from the ANTS Platform servers.

Post the results back

Your service needs a project API key for the same project. It reads the dataset and records each result as a run item. With the Python SDK:

python

from ants_platform import AntsPlatform ants = AntsPlatform() def run_remote_experiment(request_body: dict) -> None: config = json.loads(request_body["payload"] or "{}") dataset = ants.get_dataset(request_body["datasetName"]) run_name = f"remote-{config.get('model', 'default')}" for item in dataset.items: if item.status != "ACTIVE": continue with ants.start_as_current_span(name="remote-experiment-item", agent_name="support-agent"): with item.run(run_name=run_name, run_metadata=config) as span: output = my_app(item.input, config) span.update(output=output) ants.flush()

Without the SDK, create a trace with the ingestion API and then link it with POST /api/public/dataset-run-items, passing runName, datasetItemId and traceId.

LLM-as-a-Judge evaluators that target experiment runs on the dataset score the new run items automatically.

View the results

The run appears in the dataset's runs table like any other run. Select it, or compare it with other runs. See Comparing runs.

ℹ

Remote experiments only start your pipeline. ANTS Platform does not track whether the pipeline finishes. If no run appears, check your service's logs.