Metrics

Metrics are functions for measuring model performance. They might compare predicted values to ground truth, measure distributional shift, evaluate model fairness, surface explainability trends, track feature distributions, inference volumes, or anything else you can imagine. Metrics are a foundational part of evaluating and exploring your models. Arthur’s powerful Metrics API gives you the defaults you need to hit the ground running and the flexibility to define model performance however it best suits your business.

Arthur’s metrics are defined as template queries in our Query API format. These template queries are evaluated with your specified parameters, filters, and rollup. For example, when you’re viewing a Feature Drift chart in the UI, behind the scenes, the Arthur dashboard is:

• evaluating the Feature Drift metric for your model
• specifying your selected Drift Metric parameter (e.g., “PSI”) and your specified Drift Attribute parameter(s) (e.g. “Age” and “FICO Score” input attributes)
• specifying your selected timeframe as a filter over the “inference_timestamp” field
• specifying your selected rollup of “day,” “hour”, etc., to determine the granularity of the graph (or “batch_id” for batch models)