Docs/Llmops/Scores

Scores

A score is a value attached to a trace, an observation, a session or a dataset run. Scores carry a name, a value, a data type and an optional comment.

ℹ

The Scores page is not in the sidebar today. Open it at /project/<projectId>/scores. Scores also appear on each trace, observation and session, and as columns in the tracing tables. Define score configs in Project Settings > Scores / Evaluation.

Data types

Type	Value	Example
`NUMERIC`	A number	`helpfulness = 0.82`
`CATEGORICAL`	A string label	`sentiment = "positive"`
`BOOLEAN`	`1` or `0`	`passed = 1`

Where scores come from

Source	How
API	Your code sends scores with the SDK or `POST /api/public/scores`, for example user feedback or a rule based check.
Evaluation	An LLM-as-a-Judge evaluator writes a numeric score between 0 and 1 with its reasoning as the comment.
Annotation	A person adds a score from the Annotate button on a trace, observation or session, or in an annotation queue.

Send scores from code

Score the trace or observation you are in:

python

from ants_platform import get_client ants = get_client() with ants.start_as_current_span(name="answer", agent_name="support-agent") as span: answer = "Your order shipped yesterday." span.update(output=answer) span.score(name="contains_tracking_link", value=0, data_type="BOOLEAN") span.score_trace(name="tone", value="friendly", data_type="CATEGORICAL")

Or score any trace later by ID, for example when a user clicks thumbs up:

python

ants.create_score( trace_id="<trace id>", name="user_feedback", value=1, data_type="BOOLEAN", comment="Thumbs up", ) ants.flush()

Pass observation_id= to score one observation, or session_id= to score a session.

With the REST API:

bash

curl -u "pk-ap-...:sk-ap-..." \ -X POST "https://<your-host>/api/public/scores" \ -H "Content-Type: application/json" \ -d '{"traceId": "<trace id>", "name": "user_feedback", "value": 1, "dataType": "BOOLEAN"}'

Score configs

A score config defines a score once so every source uses it the same way. Create one in Project Settings > Scores / Evaluation with Add new score config:

Field	Meaning
Name	The score name.
Data type	Numeric, categorical or boolean.
Minimum and Maximum	Optional bounds for numeric scores.
Categories	Label and value pairs for categorical scores.
Description	Optional guidance shown to annotators.

Annotation queues and the Annotate drawer use score configs. Pass config_id= from the SDK, or configId in the API, to validate a score against a config. Configs can be archived, but not deleted.

Viewing scores

The Scores page lists every score with filters for name, source, data type, value and time.
Trace, observation and session views show their scores.
Score columns appear in the tracing tables, and you can filter traces by score.
Dataset runs show aggregate scores per run.
Custom dashboards can chart numeric and categorical scores.

Among the built-in roles, only Owner can create score configs or add and edit scores in the app today. See Members & Roles.