Arthur Scope FAQ

1. Can I use Arthur Scope without using the Python SDK?

Yes! The Arthur Scope platform is API-first. You can use our Arthur API to onboard models, send predictions, and query metrics and insights.

2. Does Arthur need a copy of my model?

Arthur doesn’t generally need access to your actual model but only captures the inputs to the model and the predictions it makes. This means that you can even use Arthur to monitor models you cannot access, such as models hosted by third-party services.

To enable explainability, Arthur does need access to your model. When enabling explainability, you will need to provide access to the modelspredict function.

3. What if my data is proprietary? Can I still use Arthur?

Yes! Arthur offers on-premises installation for customers with data security requirements. By integrating Arthur into your business's on-premises stack, you can be confident that all security requirements are met while still getting the benefits of the computation and analytics Arthur provides.

4. What if I don’t have ground truth labels for my data? Or what if I will have the ground truth labels in the future, but they are not available yet?

You don't need ground truth labels to log your model's inferences with Arthur.

If your ground truth labels become available after your model's inferences, whether seconds later or years later, Arthur can link these new ground truth values to your model's past predictions, linking the new values by ID to their corresponding inferences already in the Arthur system.

In the meantime, Arthur’s data drift metrics can offer leading indicators of model underperformance to keep you covered if your ground truth labels are delayed or never become available.

5. I got an error using the SDK. What do I do?

If the error message says "an internal exception occurred, please report to Arthur" that means there was a problem on our side. Please email the Arthur support team at [email protected] to let us know what happened.

Otherwise, the error message should provide helpful instructions for how to resolve the issue. If you don’t find the error message actionable, please let Arthur know so we can improve it.

6. Do I have to type my credentials in every time I use the SDK?

No! Instead of manually entering them, you can specify an ARTHUR_ENDPOINT_URL and ARTHUR_API_KEY environment variable to be used to create the ArthurAI connection object.

7. What are streaming and batch models?

Streaming and batch are two model types with different patterns of ingesting data to send to Arthur.

A streaming model processes data as a stream of individual inferences: data is logged with Arthur directly as individual inferences when the data flows into the model.

A batch model processes data as a sequence of grouped inferences, which are usually grouped over time: data is logged with Arthur as a group of inferences as the model processes the batch.

8. Which drift metric should I use?

Population stability index (PSI) is typically a good default drift metric.

In some cases, one wants a drift metric with a certain property, e.g., using a drift metric with the unit nats for interpretability or using a drift metric bounded between 0 and 1 so that drift values don't increase arbitrarily for outliers. In these cases, other metrics may be preferable to PSI.

For a review of the data drift metrics Arthur offers and their properties, see the data drift section of our Glossary. Furthermore, see our blog post for an overview of data on how Arthur automates the choice of thresholding for drift metrics.