Another common concern for organizations is blocking Personal Identifiable Information (PII) going in and out of their LLM systems. While mitigating the exposure of PII is typically the best practice all around, the two most common scenarios for teams choosing to block PII are:

1. Detecting PII in the Prompt - If your application uses a public or externally-hosted LLM, all data sent to that model may be fed back into the training data or shared across contexts with other users. In this case, you want to ensure that your user is not accidentally including their PII in the prompts to the LLM.
2. Detecting PII in the Response - Teams develop systems to add data that contains PII as context to LLMs (through fine-tuning or retrieval) to maximize the types of questions that the LLMs can answer. However, your end users may not have the right data permissions to view that PII data granularly. In this case, your application should not return information considered PII to that end user.

Types of PII

PII is information that can identify an individual when used alone or with other relevant data. PII can include:

• Direct identifiers that can identify a person uniquely, such as SSN, Passport Information, Phone Number, etc.
• Quasi-identifiers that can be combined with other quasi-identifiers to successfully recognize an individual, such as Name and Date of Birth.

The Arthur Approach

Arthur Engine checks prompts and responses for PII information via predefined PII recognizers leveraging Named Entity Recognition, regular expressions, rule based logic and checksum with relevant context. Out of the box we cover identification of the below with the option to customize it to your needs.

Currently Supported Entities

PII Customization

On top of the out of the box PII Rule users have the option to specify additional configuration for the following:

• PII entities to exclude for evaluation
• An allow list to explicitly allow certain string values that would otherwise be flagged as PII

Teams can also create custom regex-based rules for their unique PII (such as internal account IDs). Please contact Arthur Support if there are additional PII entities that you would like coverage for.

Requirements

Arthur Shield validates PII rules with either the Validate Prompt or Validate Response endpoint. While we typically recommend testing for PII in both prompt and response, there are situations where you would choose to check only one endpoint.

Enabling Governance

PII checks are some of the most common checks teams begin to enable themselves when implementing LLMs. One of the key differences we've seen with teams utilizing Shield is stronger governance into all of the blocked patterns across the organization (globally or by use-case).

Required Rule Configurations

No additional configuration is required for the default PII detection rule. For more information on how to add or enable/disable the PII Data Rule by default or for a specific Task, please refer to our Rule Configuration Guide.