Versioning is crucial for production systems in machine learning as it allows for the following:

• seamless updates and rollbacks of machine learning models
• enables performance monitoring and A/B testing, facilitating data-driven decisions for model improvements.
• and ensures reproducibility and auditability of deployed models, meeting compliance requirements and providing transparency in machine learning systems.

Model Groups == Model Tasks

We can think of versions relating to a particular model task (or use case). For example, a “Real Time Transaction Fraud Model” will be a years-long project with much evolution. Many small and large changes may occur over the years, from retraining with new data to building completely new architectures.

When you represent these new versions in Arthur Scope, they go into a model group. Within Arthur, a model group refers to a group of versions of the models that are used for the same model task.

Creating a New Model Version with the Python SDK

New model versions within Arthur are completely new ArthurModel Objects. This means that building out a new model version consists of building a new ArthurModel object and linking it to your existing ArthurModel object through its Arthur model group. That may seem a bit confusing, so we'll break the steps down below:

Step 0: Have an Initial Model on Arthur

To put the first version of your model onto Arthur, you do not need to specify its version. You just need to follow the Creating Arthur Model Object section of the documentation.

Get the Model Group for ArthurModel you want to version

Once you have been running that version of your model on Arthur for a while, you may decide that it is time to onboard a new model version. To ensure that this new version is connected with the old version, you will use the model_group_id.

Here is a code example of how to grab the model_group_id from a running ArthurModel object.

# get the model group for v1 (we will save v2 in the same model group)
model_group = connection.get_model_group(arthur_model_v1)

Build a new ArthurModel Object for this version

Build a new Arthur model object for this version, following the same steps as Creating Arthur Model Object. Just make sure that you do not save the model object until you do the next step.

# Register V2 of the model with Arthur

arthur_model_v2 = connection.model(display_name="Magic_Model", # The model name can be the same or different
                                input_type=InputType.Tabular, 
                                output_type=OutputType.Multiclass)

# creating mapping from predictions to ground truth
pred_to_ground_truth_map = {}
for i, name in enumerate(pred_df_2.columns):
        pred_to_ground_truth_map[name]= i

arthur_model_v2.build(ref_df_2, 
                   ground_truth_column='label',
                   pred_to_ground_truth_map=pred_to_ground_truth_map)

arthur_model_v2.get_attribute('label').set(categorical=True, categories=list(ref_df_2['label'].unique()))

arthur_model_v2.review()

Save your new version

The connection between your old ArthurModel object and the new ArthurModel object happens when you save the model. Here we can see how we are assigning our new model object the same model_group_id as our original ArthurModel object. We can also provide a version_label which will represent what this new version is in the platform.

When we finish linking all that, we can save the model to the platform.

# Add v2 of the model to the same model group as v1
arthur_model_v2.model_group_id = model_group.id

# Assign a version label to v2 of the Model
arthur_model_v2.version_label = "V2"

# Save v2 of the model to Arthur
model_id = arthur_model_v2.save()

Update Inference Sending Techniques

With all that done, your new ArthurModel object is now on the Arthur platform. However, you are not done just yet. You need to ensure that any system you set up for sending inferences to your old model (or want to set up for sending inferences to your new model) is implemented to begin effectively monitoring.

To do this, teams should follow the techniques listed in Sending Inferences. Know that for model versioning, teams often follow a few different patterns:

• Replacing Old Inference Connection to New Model ID: If you are switching over completely to this new model version, replacing the old ArthurModel ID connection with this new version's model id can be the easiest.
• Setting Up Validation in Production Pipelines: Many teams use versioning before completely switching which model is being served. Teams may run different types of validation tests in production (such as A/B testing, shadow deployments, or canary tests). These are all possible within Arthur, you will just need to create different ArthurModel objects for each test and set up inference sending to the platform for them to be effective.