PII redaction proxy
Notes on filtering structured identifiers and named entities before external model requests.
This prototype places a text filter before an external model endpoint. Pattern rules identify structured values; named-entity recognition adds candidates such as people and organizations.
No detection-coverage results are published here. The filter cannot guarantee that a request contains no sensitive information.
Replacement handling
Replace matched values with stable placeholders within a request. Keep any reconstruction map local, restrict access to it, and give it a short lifetime.
Define the filter’s behavior for overlapping matches, repeated values, Unicode, and partially masked identifiers. Preserve only the structure the task needs.
Logs need the same attention as the outbound request. Avoid recording the original payload in proxy errors, traces, or model-client debugging.
Failure handling
Treat an unavailable filter as a failed request. A fallback that sends the original text bypasses the control.
Restrict outbound access so applications cannot silently call the model provider directly. The proxy only covers traffic that passes through it.
Test cases
Use synthetic fixtures for identifiers, names, spelling variants, encoded text, and sensitive values split across fields. Record missed values and unnecessary redactions separately.
Check downstream task accuracy after replacement. Removing context can change an answer even when the output remains well formed.
Attachments, images, audio, and tool responses need their own handling. A text-only filter does not cover those inputs.
Technologies: Regex, Named-entity recognition, HTTP proxy.