PII Leakage

Blocking personally identifiable information (PII) from flowing in and out of LLM systems is a common requirement. Limiting PII exposure is good practice in general, but the two scenarios that most often drive teams to enforce it are:

1. PII in the prompt. If your application uses a public or externally hosted LLM, everything sent to that model may be retained, used for training, or exposed across contexts. You want to make sure users are not including their own or others' PII in prompts.
2. PII in the response. Teams add data containing PII to LLMs as context (through retrieval or fine-tuning) so the model can answer more questions. But the end user may not have permission to see that PII, and the application should not return it to them.

Types of PII

PII is information that can identify an individual on its own or in combination with other data. It includes:

• Direct identifiers that uniquely identify a person, such as an SSN, passport number, or phone number.
• Quasi-identifiers that identify a person when combined, such as name and date of birth.

The GenAI Engine approach

The PII rule checks prompts and responses with predefined recognizers that combine named entity recognition, regular expressions, rule-based logic, and checksums where they apply (for example, credit card and IBAN validation). The engine uses Microsoft Presidio for pattern and checksum recognizers and the GLiNER PII model for entity recognition. Both run locally inside the engine; no external LLM is called.

The entities below are detected out of the box, with options to customize which ones are enforced.

Currently supported entities:

Customization

On top of the out-of-the-box rule, you can configure:

• disabled_pii_entities: entity types to leave out of evaluation
• allow_list: specific string values that should never be flagged even if their entity type is enabled, such as your company's own domain

For identifiers unique to your organization, such as internal account IDs, create a regex rule. Contact Arthur Support if you need coverage for additional PII entity types.

Rule results

When PII is found, the result lists every matched span with its entity type and the recognizer's confidence.

{
  "id": "7a1c0b3e-5c2d-4f8e-9a1b-2c3d4e5f6a7b",
  "name": "PII",
  "rule_type": "PIIDataRule",
  "scope": "default",
  "result": "Fail",
  "latency_ms": 85,
  "details": {
    "message": "PII found in data: EMAIL_ADDRESS, US_SSN",
    "pii_entities": [
      { "entity": "EMAIL_ADDRESS", "span": "[email protected]", "confidence": 1.0 },
      { "entity": "US_SSN", "span": "123-45-6789", "confidence": 0.85 }
    ]
  }
}

Requirements

The PII rule runs on validate_prompt, validate_response, or both. We generally recommend checking both, though some use cases only need one.

Governance

PII checks are usually the first rules teams enable when they adopt LLMs. Configuring them as default rules in the GenAI Engine gives you one place to see and manage which PII patterns are blocked across the organization, globally or per use case, rather than re-implementing the check in each application.

Configuration

No additional configuration is required for the default PII rule. The rule type is PIIDataRule, and apply_to_prompt and apply_to_response can each be true or false. See the Rule Configuration Guide for the request body, the optional disabled_pii_entities and allow_list settings, and enabling or disabling the rule by default or per task.