What goes into a cache key, and what happens when two requests that should get different responses collide on one?

The key is a claim about identity

A cache is a map from key to response, and storing an entry asserts something quite strong: that any future request producing this key can be served these bytes. Every caching bug that is not about staleness is a violation of that assertion. Two requests that should have received different responses produced the same key, so one of them received the other's answer.

That framing is worth stating out loud, because it turns key design from a naming exercise into a derivation. You do not invent a key; you enumerate everything about a request that can change the response, and the key is that list. Anything you omit is an assertion that it does not matter, and you had better be right, because the cache will not check.

Enumerating what varies

Start from the response and work backwards. For a product page, the response might change with the product identifier, the locale used for copy, the currency used for price, the country, because availability and tax differ, the device class, if you render different image sets, whether the viewer is signed in, since a signed-in viewer may see stock reserved for them, and which experiment arm they are in. That is seven inputs, and only one of them is in the path.

The other six arrive as headers, cookies, negotiated content, or state computed server-side from a session. This is the asymmetry that catches people: the natural key is the URL, and the URL usually carries a minority of the inputs that matter. Personalisation, internationalisation and experimentation all add inputs off the URL, which is precisely why they are the three features that most often break a cache that worked fine before them.

The counter-pressure is that every input you add to the key multiplies the number of stored entries and divides your hit rate by the same factor. Four locales times three currencies times two device classes times four experiment arms is ninety-six variants of one page, and if your traffic to that page was thin to begin with, each variant now expires before it is hit twice. So the design question is not "what could vary" but "what varies, and can I collapse it". Collapsing is legitimate work: normalise a long tail of user-agent strings into two device classes, derive currency from country rather than keying on both, and move genuinely per-user fragments out of the cached document so that the document itself stays cacheable for everyone.

Vary is a weak instrument

HTTP's mechanism for this is the Vary response header, which names the request headers a shared cache must include in its key. It is correct and it is standardised, and it disappoints in practice for two reasons.

The first is granularity. Vary: Accept-Encoding works well because the value space is small and effectively normalised by convention. Vary: User-Agent is close to a cache bypass, because the value space is enormous and near-unique per client, so you end up with a hit rate near zero that still costs you the storage. Vary: Cookie is the same problem with worse consequences, since every distinct cookie jar becomes its own entry and one analytics cookie fragments the whole route.

The second is that Vary cannot express anything that is not literally a request header. Experiment arm, tenant, entitlement tier and country-derived-from-address are all computed, and a shared cache has no way to compute them. The practical answer is to make the derived value explicit: compute it at the edge, write it into a small normalised header or into the key configuration of your CDN, and key on that rather than on the raw input it came from. You have then chosen the cardinality yourself instead of inheriting whatever the client happened to send.

Normalisation must have exactly one owner

Query strings are where key spaces quietly explode. ?a=1&b=2 and ?b=2&a=1 are the same request and two keys. A tracking parameter that never reaches your application still splits the cache, and a campaign that appends a unique click identifier to every link gives you a hit rate of zero and a traffic graph that looks like success. So sort the parameters, drop the ones the origin ignores, lower-case what is case-insensitive, and decide whether a trailing slash is significant.

The danger is that normalisation ends up implemented twice, once at the CDN and once in the application's own cache, and the two drift. When they disagree, the layers hold entries under different keys for the same logical resource, so a purge issued against one leaves the other serving the old bytes — a stale response that no amount of invalidation clears, because you are invalidating a key nobody stored. Write the rule once, as configuration or as a shared library, and derive both layers from it.

When an omitted input becomes an attack

Treating this purely as a correctness problem understates it, and the security framing is what distinguishes a senior answer.

If your application reflects a request header into the response — a host header used to build absolute links, a forwarded-protocol header used to build a redirect — and that header is not in the cache key, then an attacker sends one request carrying a hostile value, your origin renders it, and the shared cache stores the poisoned response under the ordinary key. Every subsequent visitor is served the attacker's content from your own domain, with your certificate on it. The requirement is therefore absolute: any input that can reach the response body or headers is either in the key, or is not permitted to influence the response at all.

The mirror image is cache deception. A signed-in user's personalised page is served from a URL that the CDN classifies as static, because a rule caches by file extension or by path prefix rather than by what the response says about itself. The origin returns a private page, the edge stores it under a URL an attacker can guess, and the next request for that URL returns somebody's account details. The structural defence is that authenticated responses declare themselves: Cache-Control: private for anything user-specific and no-store for anything sensitive, applied by default in the response layer rather than added per endpoint by whoever remembers. A caching rule that keys off the request path can be fooled by the path; a rule that respects response directives cannot be.

Finding a collision after it has happened

The unpleasant property of a mis-keyed response is that it is a valid response. It is well-formed, it returns 200, it satisfies every schema check, and it simply belongs to somebody else. Error-rate and latency dashboards show nothing at all, and the report arrives from a customer who saw a name that was not theirs.

Three things make this detectable. Carry an identifier for the keyed dimensions inside the response — the tenant, the locale, the experiment arm — and have the client or a synthetic check assert that it matches what was requested, so a mismatch becomes a hard signal rather than an anecdote. Emit the computed key alongside hit-or-miss status in your access logs, so that after a report you can ask which key served that response instead of reasoning about it from first principles. And watch cardinality per route: a sudden rise in distinct keys usually means a new input has leaked into the key, and a sudden fall means one has dropped out of it, which is by far the more dangerous direction.

Derive the key by listing every input that can change the response, then collapse that list deliberately to protect your hit rate. Anything you leave out is a promise that it cannot affect the answer, and a shared cache will enforce that promise on your users whether or not it is true.