> ## Documentation Index
> Fetch the complete documentation index at: https://tracevell.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Compare evaluation runs

An evaluation in isolation tells you your current score. What you actually want to know is whether the score moved. That's what groups, the progression chart, and side-by-side comparison are for.

## Group runs to compare them

Pass `groupName` / `group_name` to `evaluate()`. Every run with the same group name lands together on the evaluations page.

> [tab] TypeScript
>
> ```typescript theme={null}
> evaluate({
>   data,
>   executor,
>   evaluators,
>   name: 'Capitals v2 (harder countries)',
>   groupName: 'capitals',
> });
> ```

> [tab] Python
>
> ```python theme={null}
> evaluate(
>     data=data,
>     executor=executor,
>     evaluators=evaluators,
>     name="Capitals v2 (harder countries)",
>     group_name="capitals",
> )
> ```

Pick one name per *thing you're testing*, not per *version*. The capitals eval stays in the `capitals` group whether you're swapping models, prompts, or datasets. Changing the group name means Tracevell can't chart the runs together.

## Read the progression chart

The evaluations page lists every group in a sidebar, with a run count and the time of the latest run. Select a group and the right side shows its progression chart and run list, newest first.

![Evaluations page with the capitals group selected in the sidebar and a progression chart showing accuracy flat at 1.0 and length\_ok falling to 0.0](https://mintcdn.com/laminarai/O81ZcDOMmMmrw7-x/images/evaluations/eval-list.png?fit=max\&auto=format\&n=O81ZcDOMmMmrw7-x\&q=85\&s=aa8062432d14770edc6230319f3f6cae)

Three runs in the capitals group. accuracy holds at 1.0 across every run; length\_ok drops to 0.0 on the most recent run

One line per score dimension. Each point is one run's aggregate for that dimension. A sudden drop on one line means a regression on that dimension.

A few controls worth knowing:

* **Aggregation dropdown** next to the group name. Average by default; switch to Sum, Min, Max, Median, p90, p95, or p99. Percentiles are the right choice when a few catastrophic rows hide behind a healthy average.
* **Legend toggles**. Click a score name in the chart legend to hide or show that line. Useful when one dimension's scale dwarfs another's.
* **Eye icons** on run rows. Click the eye to exclude a run from the chart, for example a half-finished run or a known-bad experiment that distorts the y-axis.

In the screenshot above, `length_ok` fell from 1.0 to 0.0 on the most recent run because the prompt was changed to ask for a one-sentence fun fact instead of a one-word answer. Every output now exceeds the 50-character limit the evaluator checks for.

## Side-by-side comparison

Click any run to open its detail page, then use the **Select compared evaluation** dropdown in the header to pick a baseline run from the same group. The comparison is in the URL (`?targetId=<evaluation-id>`), so you can share the link. Hit **Reset** to drop the baseline.

With a baseline selected:

* The score card shows `old → new` with a colored delta badge, for example `0.83 → 0.74` with `▼ 11.5%`. Switch the score dropdown to see the delta for each dimension.
* Every row in the datapoint table shows its own `old → new` delta per score column, so you can see exactly which datapoints moved.

Rows are matched across runs by [task id](https://tracevell.com/docs/evaluations/concepts.md#tasks-and-trials), not by position. A run that only executes a subset of the dataset still lines up with a full run, and a Task with several Trials compares its average against the baseline Task's average.

> [!NOTE]
>
> Runs recorded before task ids existed use the row's index as the task id. They still compare with each other, but not with newer runs whose task ids come from a dataset or are set explicitly.

![Comparison view with the score card showing 0.83 to 0.74 and a datapoints table with per-row score deltas](https://mintcdn.com/laminarai/O81ZcDOMmMmrw7-x/images/evaluations/eval-compare.png?fit=max\&auto=format\&n=O81ZcDOMmMmrw7-x\&q=85\&s=7a36b9ac1c01ea456652f2d287508591)

Comparing a text-to-SQL run without a style guide against the baseline with one. token\_f1 drops from 0.83 to 0.74 (-11.5%); per-row deltas show which questions regressed

The score card tells you the direction. The per-row deltas tell you exactly which datapoints moved. The trace panel on the right shows you *why*: click a regressed row and read what the model actually produced.

## See why a score moved with a render template

Per-row deltas point at the regressed datapoints, but the answer to "what actually changed" is buried in the executor's output. For evals where the output is comparable to the target (SQL generation, structured extraction, translation), a [trace render template](https://tracevell.com/docs/platform/render-templates.md#trace-templates) turns the trace panel into a purpose-built diff view.

The example below is a text-to-SQL eval. The template pulls the question from the executor's input, the generated SQL from its output, and the target SQL from the evaluator's input, then renders a word-level diff:

![Evaluation trace rendered through a custom SQL diff template showing model output and target side by side with highlighted differing tokens](https://mintcdn.com/laminarai/XAojCVbhs9bn1a75/images/evaluations/eval-custom-renderer.png?fit=max\&auto=format\&n=XAojCVbhs9bn1a75\&q=85\&s=995ff2aad869c57f76fae6a42f21d5f0)

A trace render template on an eval trace: the question, both scores, and a word diff between the model's SQL and the target. Red tokens exist only in the output, green only in the target

Reading this view across a few rows answers the regression in seconds: without the style guide, the model stopped aliasing tables and switched join strategies, so `token_f1` dropped even where the SQL was semantically fine.

To build one, open any datapoint's trace, click the view dropdown (where **Transcript** and **Tree** live), pick **+ New template**, and describe the view: Tracevell generates both the span filter (here `span_type IN ('EXECUTOR', 'EVALUATOR')`) and the JSX from the trace's real spans. Once created, the template appears in the same dropdown for every trace in the project, and the pane remembers your pick, so clicking through datapoints keeps the diff view open.

The full walkthrough, template source, and span payload shape are on the [Custom rendering](https://tracevell.com/docs/evaluations/custom-rendering.md) page.

## Filter by group in the list

The evaluations page groups runs by default. Click a group in the sidebar, or visit `/evaluations?groupId=<group-name>` directly. The progression chart and run list scope to the selected group.

## Export comparisons

Hit **Download** on the evaluation detail page to export the datapoints, scores, and executor outputs as CSV. Useful for external analysis or for building regression test suites out of the rows that failed.

For anything beyond CSV, query the underlying table with SQL:

```sql theme={null}
SELECT
    evaluation_id,
    AVG(scores['accuracy']) AS avg_accuracy,
    AVG(scores['length_ok']) AS avg_length_ok
FROM evaluation_trials
GROUP BY evaluation_id
ORDER BY avg_accuracy DESC
```

The SQL editor exposes `evaluation_trials` (one row per Trial, with `task_id`, `scores`, `target`, `executor_output`, and the joined trace; `evaluation_datapoints` is an alias). Each row carries `evaluation_name`, and the `evaluations` table holds each run's group and metadata. Group by `task_id` within an evaluation to average a Task's Trials. See the [SQL editor](https://tracevell.com/docs/platform/sql-editor.md) page for more.

## Next steps

> [card] [Render templates](https://tracevell.com/docs/platform/render-templates.md)
>
> Build custom trace views like the SQL diff above.

> [card] [Datasets](https://tracevell.com/docs/evaluations/datasets.md)
>
> Keep the dataset constant across runs so comparisons are apples-to-apples.

> [card] [SQL editor](https://tracevell.com/docs/platform/sql-editor.md)
>
> Query `evaluation_trials` for bespoke comparisons and dashboards.

> [card] [SDK reference](https://tracevell.com/docs/sdk/evaluations.md)
>
> Full parameters for `evaluate`, `LaminarDataset`, and `EvaluationDataset`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.