You cached something in a module-level dictionary. It works on your laptop and behaves oddly in production. What is different?

The development server is one process; production is not

Running manage.py runserver or uvicorn app:app gives you a single process with a single copy of every imported module, so a module-level dictionary behaves like the global variable you intended. Production runs a process manager with several workers, each of which is a separate interpreter that imported your code independently. There are now four dictionaries, not one, and the load balancer or the operating system decides which one a given request talks to.

Nothing about this is subtle once you see it, but the symptoms are confusing because they are statistical rather than deterministic. A value written by one request is found by roughly one request in four. A rate limiter allows four times the configured rate. A feature flag toggled through an admin endpoint appears to work, then appears not to, depending on which worker served the page. A session stored in memory logs the user out intermittently. And a cache hit ratio measured in production is far worse than the one measured locally, because each worker has to warm its own copy.

Per-process, per-thread, per-loop

Being precise about the scope of the state is what separates a good answer here, because there are three levels and they behave differently.

Module globals are per process. Two workers share nothing at all — not memory, not locks, not connections — so no amount of locking in your code coordinates them. A threading.Lock around that dictionary makes it safe within one worker and does absolutely nothing across workers, which is a genuinely common misdiagnosis.

Within a worker, whether you also need that lock depends on the concurrency model. A worker serving requests on threads shares module globals between them, so a read-modify-write on your dictionary is a real race despite the GIL, which only guarantees that individual bytecodes do not interleave. A worker running an asyncio event loop shares the globals across all coroutines on that loop, and although no two coroutines run simultaneously, any await inside a read-modify-write sequence is a point at which another coroutine can observe or change the half-updated state. That is the async equivalent of the same bug and it is harder to see, because the code contains no threads.

Request-scoped state is the fourth case and the one people reach for a global to avoid. It belongs in the request object or in a dependency the framework resolves per request, and if it genuinely must be ambient, contextvars is the mechanism that keeps a value scoped to the current task rather than to the process.

Startup hooks run once per worker, not once

The same arithmetic breaks anything you attached to application startup. A lifespan handler, a Django AppConfig.ready, or a @app.on_event("startup") callback runs in every worker, so four workers mean four executions.

Usually that is harmless — opening a connection pool per worker is correct — but two cases are not. An in-process scheduler started at boot now has four copies firing the same job on the same schedule, which produces duplicated emails, duplicated charges and rows written four times. And a migration, a cache warm or a data backfill placed in a startup hook runs concurrently in four processes racing each other. Scheduled work belongs in something that runs once by construction: a dedicated single-instance process, the platform's own scheduler, or an advisory lock in the database that only one worker acquires.

Preload, fork and the sharing that erodes

Gunicorn's preload option changes where the import happens: the application is loaded in the master process and workers are forked from it, so globals are initialised once and the children start with a copy. It is worth knowing because it explains two behaviours. Objects you must not share across a fork, connections in particular, are now inherited and must be recreated per worker or they will be used by several processes on one socket. And a large read-only structure loaded before the fork does start out shared, because the pages are copy-on-write.

The saving is real but it decays. CPython updates a reference count on every object it touches, and that write dirties the page the object header sits on, so merely reading a Python data structure gradually un-shares it. A preloaded dictionary of a million entries is not permanently free memory across four workers; it converges towards four copies as each worker touches it. This is why in-memory data of any size belongs in something outside the process, and why the honest answer to "we preload so it is only in memory once" is "at first".

Where the state belongs instead

The decision rule is short. If correctness depends on every worker seeing the same value, it cannot live in the process: put it in Redis, the database, or another shared store, and let all workers read the one copy. If it is a pure function of inputs and stale copies are merely wasteful, a per-process cache is fine, provided it is bounded and given a time-to-live so a restart is the only thing that has to bring it back into line.

The Django-specific version of this trap is worth naming, because the framework offers a cache API that looks shared and is not: the local-memory backend keeps a private cache per process, so it behaves exactly like your dictionary. Configuring a shared backend changes the semantics without changing any calling code.

Finally, remember that all process-local state is also deployment-local. A rolling restart replaces every worker, so anything held in memory vanishes, and any behaviour that depended on it having been warm — a rate limit, a de-duplication window, a computed lookup table — resets at the exact moment the system is under the most change.

Module-level state is per worker process, and locking cannot make it shared. Decide whether every worker must agree: if so it belongs in a shared store, and if not, bound it, expire it, and expect a deployment to empty it.