How to Set Up and Track Metrics by Version

Steps to Enable Model Versioning

1. Add a model version column
   Add a model_version column to the dataset(s) used by your model.
2. Configure as a segmentation column
   Set model_version as a segmentation column for all metrics attached to the model.

3. Run a metrics calculation job
   This ensures that the database contains metric data with the new segmentation column.
4. Update or create a custom dashboard
   1. In your queries, select the model version from dimensions:
      1. dimensions->>'model_version'

         select time_bucket(interval '1 day', timestamp) as bucket,
                sum(value)                               as total,
                dimensions ->> 'prompt_version_id'       as prompt_version_id
         from metrics_numeric_latest_version
         where metric_name = 'inference_count'
             AND timestamp BETWEEN '{{dateStart}}' AND '{{dateEnd}}'
         group by bucket, prompt_version_id

      2. Add model_version to the GROUP BY clause in each query.
5. Add segmentation in charts
   In the Chart Builder, click Add segmentation and select the model_version column for each chart in your dashboard.

Once completed, your metrics will begin populating by model version.`

Alternative Setup
Instead of adding a model_version column, you can:

1. Create a new dataset for each model version.
2. Add the dataset to the model.
3. Use dataset_id in the dimensions field.

Follow steps 4–5 above to group and segment by dataset_id.