Product DocumentationAPI and Python SDK ReferenceRelease Notes
Schedule a Demo
Schedule a Demo

Test Explainability Locally

It can be useful to test explainability locally before enabling explainability on Arthur. To do this, teams need to use the ExplanationPackager class.

from arthurai.explainability.explanation_packager import ExplanationPackager

Next, you create a packager object using the exact same parameters you use when enabling explainability for your model. This allows you to test your explanations locally first.

## Since we are testing on the Arthur model where we enabled whitsepace, we do not need to speficy the text delimiter 
packager = ExplanationPackager(
    #text_delimiter = '\s+'

After defining your packager, there are two models that need to be run to be able to run an Arthur Explainer locally.

## Create this packager object 

## Take the explainer from this packager to run locally 
arthur_explainer = packager.explainer

After creating your explainer, you need to test it on a small sample of data to ensure that explanations are created. When creating this sample function, you should also make sure that you delete any attributes that your model does not use to make predictions (i.e. are not ModelPipelineInputs)

Note: These attributes do not need to be deleted to run explainability in the Arthur platform. The Arthur platform will run explanations with ModelPipelineInputs automatically, however, when running the explainer locally this needs to be done.

## Take in Example Inference Data 
## In practice, commonly sampled from the reference dataset for testing 
path_to_example_data = 'test.csv'
training_data = pd.read_csv(path_to_example_data, nrows=100)

def get_sample(num_samples):
    sample_inf = training_data.sample(num_samples)
    ## Delete Any Non Input Attributes 
    del sample_inf['lessthan50K']
    del sample_inf['greaterthan50K']
    del sample_inf['target']
    del sample_inf['sex']
    del sample_inf['race']
    return sample_inf.values.tolist()
sample = get_sample(1)

Finally, we are going to take our sample and run it through the explanation generator.

Key Thing Of Note: This call will match the model task type and explainability type that was specified during the original packager call. For this example, as we will see below, that is explain_shap_tabular.
However, for other model types and explanations you may want to use:

  • explain_tabular_lime
  • explain_nlp_lime
  • explain_image_lime

When calling any of these functions, we need to include two parameters. The first is your sample. The second is the number of samples we want our surrogate model to generate. The default value within Arthur is 2000, however, in this example of testing explainability locally before enabling it within Arthur, we are not evaluating for accurate explanations. Instead, we are validating that explainability runs.

explanation = arthur_explainer.explain_tabular_shap([sample], num_samples=5)