Summary
A proxy layer that runs a regex pass for structured PII and a BERT-NER pass for unstructured entities, combining both before any outbound model call.
Outbound PII Filter
A security-first proxy that redacts sensitive data before prompts leave the network, using a two-pass regex and NER pipeline.
Project Brief
Summary
A proxy layer that runs a regex pass for structured PII and a BERT-NER pass for unstructured entities, combining both before any outbound model call.
Problem
Enterprise AI is hard to take seriously when PII can leak upstream by default — and neither regex alone nor NER alone catches everything.
Hypothesis
If sensitive data is filtered at the runtime boundary before model calls, more AI workflows become acceptable in security-conscious environments.
Outcome
Built and measured a two-pass redaction pipeline over 30 enterprise prompts across 10 entity types. Combined detection reached 96.6% type coverage at 84.8% context preservation. Regex catches 100% of structured types (email, phone, SSN); NER adds the full PERSON, LOC, and partial ORG coverage that regex cannot reach.
Goals
Technologies Used
The core idea is straightforward: do not trust every prompt to leave the network untouched. This project introduces an explicit boundary before external model calls, where the system inspects outbound text, redacts risky values, and replaces them with typed placeholder tokens that preserve the prompt structure without exposing the underlying data.
Two passes are required because structured and unstructured PII are fundamentally different problems. Regex handles anything that has a predictable format — email addresses, phone numbers, SSNs, credit cards, IP addresses, API keys. No model needed; the patterns are deterministic. NER handles anything that requires understanding context — person names, organization names, locations. No regex can reliably catch “Sarah Johnson” or “Goldman Sachs” without prior knowledge of what names look like in context.
The overlap between the two passes is minimal and by design. Regex does not produce false positives on PERSON; NER does not produce false positives on EMAIL. The deduplication layer removes overlapping spans by longest-match, so even the few entity types that could be caught by both do not double-count.
dslim/bert-base-NER via the transformers pipeline, fine-tuned on CoNLL-2003 (PER, ORG, LOC, MISC). Aggregation strategy: simple (merges subword tokens into full entity spans).[EMAIL], [PERSON], etc.) so character offsets remain valid during substitution.The biggest lesson was that security is not a blocker for AI. It is the thing that makes AI usable when the default trust model is not acceptable. The second lesson was complementarity: regex and NER are not competing approaches to the same problem. They cover disjoint parts of the entity space. A production system that uses only regex misses every human name and organization. A system that uses only NER misses every structured secret that does not read like a natural-language entity. The two-pass design is not belt-and-suspenders — it is the minimum required to cover the full threat surface.
01
Regex alone captures 100% of structured entity types — EMAIL, PHONE, SSN, IP_ADDRESS, and API_KEY — with zero reliance on a model. These types are fully predictable by pattern and regex handles them without false positives.
02
BERT NER handles every entity class that regex cannot reach — PERSON at 100%, LOC at 100%, ORG at 78%. Without the NER pass, all names, organizations, and locations flow through to the model unredacted.
03
Combined two-pass detection reaches 96.6% type coverage at 84.8% average context preservation. The redacted prompt retains enough structure for the downstream model to remain useful. The 22-point gap between regex-only and combined coverage is the NER contribution.
Analysis
Detection Rate by Entity Type — Regex vs NER vs Combined
Loading chart...
30 enterprise prompts across HR, Finance, Support, and IT domains. Stacked bars show regex-only detection (indigo), NER adds (green), and missed (zinc). Structured types (EMAIL, PHONE, SSN, IP_ADDRESS, API_KEY) are caught entirely by regex at 100%. Unstructured types (PERSON, LOC) are caught entirely by NER. ORG reaches 78% — the hardest entity class. Combined coverage: 96.6% across 10 entity types.
[ Connect ]
If you are trying to make AI usable inside a security-sensitive environment, this is one of the first control layers I think about.
You are reaching
John Meyer
Security Engineer → AI