A customer reports seeing another company's records in your admin console. Walk me through the first hour, and then tell me what you change so this class of bug cannot happen again.
Treat it as a confirmed breach until proven otherwise: preserve logs, scope who saw what, and start the disclosure clock alongside the investigation. The cause is almost always tenant scoping living in application code, where one forgotten filter leaks everything - so move enforcement below the code, to row-level security.
Answer
Short answer
Assume a breach until you can prove otherwise. Preserve the logs, determine exactly which records were rendered to which users and for how long, and start the disclosure clock in parallel with the engineering work. Then fix the class rather than the case: a leak like this almost always means tenant scoping is enforced in application code, where any query that forgets the filter returns everything. The durable answer is to make forgetting impossible.
Was this answer helpful?
Likely follow-ups
- The tenant id comes from a request header the client controls. What is wrong with that, and what do you use instead?
- How would row-level security have prevented this, and what does it cost you operationally?
- A background job runs with no user context. How does it get a tenant scope?
- Your cache key omits tenant id. What happens, and how would you find it?
- How do you test for this so it fails in CI rather than in production?
Related questions
- How do you isolate tenants in a shared vector index?hardAlso on multi-tenancy and tenant-isolation6 min
- Tenants share one cluster and the largest is a thousand times the smallest. How do you place them?hardAlso on multi-tenancy and tenant-isolation5 min
- Would you use iptables or eBPF for network policy enforcement in a massive multi-tenant Kubernetes cluster, and what are the operational trade-offs?hardAlso on security and multi-tenancy2 min
- One tenant bursts to ten times their normal traffic and every other customer's latency doubles. Your global rate limit was never hit. How would you design for fairness instead?hardAlso on multi-tenancy5 min