It can be useful to test explainability locally before enabling explainability on Arthur. To do this, teams need to use the ExplanationPackager class.
from arthurai.explainability.explanation_packager import ExplanationPackager
Next, you create a packager object using the exact same parameters you use when enabling explainability for your model. This allows you to test your explanations locally first.
## Since we are testing on the Arthur model where we enabled whitsepace, we do not need to speficy the text delimiter
packager = ExplanationPackager(
arthur_model,
df=df2,
project_directory=path,
user_predict_function_import_path='entrypoint',
streaming_explainability_enabled=True,
requirements_file="requirements.txt",
#text_delimiter = '\s+'
explanation_algo='shap')
After defining your packager, there are two models that need to be run to be able to run an Arthur Explainer locally.
## Create this packager object
packager.create()
## Take the explainer from this packager to run locally
arthur_explainer = packager.explainer
After creating your explainer, you need to test it on a small sample of data to ensure that explanations are created. When creating this sample function, you should also make sure that you delete any attributes that your model does not use to make predictions (i.e. are not ModelPipelineInputs)
Note: These attributes do not need to be deleted to run explainability in the Arthur platform. The Arthur platform will run explanations with ModelPipelineInputs automatically, however, when running the explainer locally this needs to be done.
## Take in Example Inference Data
## In practice, commonly sampled from the reference dataset for testing
path_to_example_data = 'test.csv'
training_data = pd.read_csv(path_to_example_data, nrows=100)
def get_sample(num_samples):
sample_inf = training_data.sample(num_samples)
## Delete Any Non Input Attributes
del sample_inf['lessthan50K']
del sample_inf['greaterthan50K']
del sample_inf['target']
del sample_inf['sex']
del sample_inf['race']
return sample_inf.values.tolist()
sample = get_sample(1)
Finally, we are going to take our sample and run it through the explanation generator.
Key Thing Of Note: This call will match the model task type and explainability type that was specified during the original packager call. For this example, as we will see below, that is explain_shap_tabular.
However, for other model types and explanations you may want to use:
- explain_tabular_lime
- explain_nlp_lime
- explain_image_lime
When calling any of these functions, we need to include two parameters. The first is your sample. The second is the number of samples we want our surrogate model to generate. The default value within Arthur is 2000, however, in this example of testing explainability locally before enabling it within Arthur, we are not evaluating for accurate explanations. Instead, we are validating that explainability runs.
explanation = arthur_explainer.explain_tabular_shap([sample], num_samples=5)