How do you implement dynamic PII masking across a highly decentralised data mesh without destroying the analytical utility of the data for downstream machine learning workloads?

Short answer

An examination of decentralised data governance, exploring dynamic masking in the compute layer, deterministic tokenisation for referential integrity, and standardised tagging.

The illusion of decentralised compliance

Decoupling a monolithic data lake into independent data domains empowers teams to treat data as a product, but it fundamentally fractures security oversight. In a data mesh, enforcing Personally Identifiable Information (PII) masking across autonomous domains creates an immediate conflict between central governance mandates and decentralised engineering agility. Security demands that unauthorised personnel never query unmasked PII, while data scientists insist that aggressive redaction obliterates the analytical utility necessary for critical workloads, such as fraud detection modelling.

Why static masked copies don't survive a data mesh

The default, lazy engineering response is to maintain static, masked copies of every data product. This approach is a catastrophic waste of storage and introduces unbearable pipeline latency. It guarantees out-of-sync datasets and creates an exploding matrix of access roles tied to physical copies, ultimately defeating the core tenets of a data mesh.

Dynamic masking at the compute boundary

The robust architecture shifts masking entirely to the compute layer. By deploying a federated query engine integrated with a central policy evaluation service, masking is applied dynamically. When a query is executed, the engine intercepts the request, evaluates the user's IAM role, and seamlessly rewrites the query execution plan to mask sensitive columns in-memory. The raw data on disk remains untouched.

Scaling this across dozens of domains requires abandoning schema-specific rules in favour of a strict, standardised metadata ontology. Domain teams must tag datasets during deployment using a unified taxonomy. Automated data discovery crawlers equipped with ML-based classifiers continuously scan the mesh, flagging untagged PII and exposing compliance violations. Dynamic policies are authored against these ontological tags, ensuring universal enforcement regardless of a domain's internal schema drift.

Preserving referential integrity with tokenisation

Simple redaction breaks cross-domain joins, rendering the data mesh useless for complex analytics. To satisfy the data science teams without violating compliance, deterministic tokenisation is mandatory. A central tokenisation service replaces sensitive identifiers with non-reversible, format-preserving tokens during ingestion or dynamically at query time. Because the tokenisation is deterministic, a customer identifier always yields the identical token across all domains, preserving referential integrity for complex joins while keeping the underlying PII hidden.

For edge cases requiring raw data, such as active fraud investigations, a strictly audited "break-glass" procedure is implemented. Highly privileged users request ephemeral role elevation through an automated workflow. Every query executed under this elevated role is meticulously logged, routed to a secure audit trail, and scrutinized for exfiltration attempts.

flowchart TD
  A["User Query (Trino/Spark)"] --> B{"Policy Engine Intercept"}
  B -->|Evaluate Role & Tags| C["Query Rewrite Engine"]
  C --> D["Fetch Data from Domain Storage"]
  D --> E["Apply Dynamic Masking/Tokenisation"]
  E --> F["Return Masked Results"]
  G["Domain CI/CD Pipeline"] -.-> H["Central Metadata Catalog"]
  H -.-> B

True data governance in a distributed architecture is not achieved by locking data away, but by injecting security policies dynamically into the compute layer, ensuring compliance scales effortlessly with the growth of the mesh.

© 2026 Preptima. Originally published at preptima.com.