Python is three fairly separate job markets — backend web, data and machine learning, and automation — that share a syntax and almost nothing else. Interviews differ sharply between them, so saying you know Python conveys much less than it sounds like.
Very high demand20 min readBackend engineer (Python), Data engineer, Machine learning engineer, Data scientist, Site reliability engineer, Automation and tooling engineer, Quantitative developerUpdated 2026-07-27
Assumes you know: Programming fundamentals: variables, loops, functions, Basic command-line use, For the data track, some statistics; for the backend track, SQL and HTTP
Overview
What this area actually covers
Python is a dynamically typed, garbage-collected, interpreted language with an
unusually large standard library and an even larger third-party ecosystem. The
language itself is small enough to learn in a fortnight. What takes years is the
ecosystem, and the ecosystem is not one thing — it splits into three markets that
happen to share a syntax.
Backend web means Django, FastAPI or Flask serving HTTP, backed by Postgres,
with Celery or a similar worker for background jobs. Data and machine learning
means NumPy, pandas or Polars, scikit-learn, PyTorch, and increasingly the
orchestration and serving layer around models. Automation, scripting and
platform work means glueing systems together: CI pipelines, cloud SDKs,
infrastructure tooling, log processing, internal command-line utilities.
These three overlap far less than outsiders assume. A Django engineer of ten
years' standing may have never opened a Jupyter notebook. A machine-learning
engineer may have no view on how to structure a REST API, and no reason to. The
skill that generalises across all three is understanding how the runtime behaves;
the libraries do not transfer.
There is one more distinction worth drawing at the outset, because it silently
governs a great deal. Python is a language specification with several
implementations, and in practice almost everyone means CPython, the reference
implementation written in C. Statements about the global interpreter lock,
reference counting and C extension modules are statements about CPython rather
than about the language. PyPy, a just-in-time compiling implementation, and the
various embedded and alternative runtimes behave differently, and the reason they
have never taken over is precisely the C extension ecosystem that the reference
implementation defines. Knowing that the implementation and the language are
separable is the difference between reciting a constraint and understanding one.
What Python is not, for hiring purposes, is the notebook-and-plot activity most
people first encounter. That is a genuine use, but it is the shallowest end of a
market where the paid work is production systems: data pipelines that must run
every night, services that must not fall over, models that must be retrained and
redeployed.
The seven subsections split along a deliberate line: the first five are about the
language and its runtime and are common to every Python job, and the last two are
about the frameworks and the practice that most Python roles are actually
measured on. Someone preparing for an interview in a hurry should work through
core Python, then concurrency, then the framework their target role uses.
Subsection
What it is for
Core Python
The data model and the semantics the syntax hides
OOP in Python
How classes really resolve, and the protocol system underneath
Decorators, Generators & Iterators
Functions as values, and laziness as a design tool
Concurrency & the GIL
Choosing between threads, processes and asyncio
Python Memory & Performance
Where the time and the memory actually go
Django & FastAPI
The two dominant server frameworks and their behaviour
Testing in Python
pytest, mocking boundaries, and what coverage means
Core Python
Core Python covers the data model: names as bindings rather than boxes,
mutability and what it implies for defaults and aliasing, scoping and closures,
truthiness, equality versus identity, shallow versus deep copying, and
comprehensions. It is a separate area because these are the semantics that every
other subsection assumes, and because they are the ones the syntax actively
disguises — a = b looks like a copy and is not, and a startling proportion of
production Python bugs are that one sentence played out at scale. Expect
questions that seem trivially easy and turn out to be probing whether you know
what an object is.
OOP in Python
Dunder methods and the protocols they implement, the method resolution order and
how multiple inheritance is linearised, descriptors, properties, class versus
instance attributes, dataclasses, and metaclasses at the far end. This exists
separately because Python's object system is protocol-based rather than
inheritance-based: an object is iterable because it implements a method, not
because it inherits from an interface, and that reverses much of the intuition
carried over from Java or C#. What you will find here is less about designing
class hierarchies — Python codebases tend to be flat — and more about how the
machinery you use every day is wired.
Decorators, Generators & Iterators
Closures, higher-order functions, decorators with and without arguments,
generators and the yield protocol, generator expressions, context managers, and
the composable utilities you build from them. It is its own area because laziness
is a distinct design tool: a generator lets you process a file larger than memory
or an unbounded stream without changing the shape of your code, and understanding
that is the difference between a pipeline that scales and one that loads
everything first. Decorators earn their place here because they are the mechanism
behind an enormous amount of framework behaviour, and reading one you did not
write is a genuine skill.
Concurrency & the GIL
Threads, processes, asyncio, and the reasoning that picks between them. The
global interpreter lock is the reason this cannot be treated as an afterthought:
what it does and does not prevent determines which of the three models will help
you, and the answer depends entirely on whether your workload is waiting or
computing. This is a separate subsection because it is the most commonly
misunderstood topic in Python interviews and the one where a memorised slogan is
most easily distinguished from understanding. You will find questions about
async/await mechanics, about when multiprocessing is worth its serialisation
cost, and about mixing blocking code into an event loop.
Python Memory & Performance
Reference counting, the cycle-detecting garbage collector layered on top of it,
object overhead, profiling with the standard tooling, and identifying where
interpreter overhead dominates versus where you are simply waiting on something
else. This area exists because "Python is slow" is a claim nobody should make
without a profile, and because the fixes are so different depending on the cause:
vectorise, move to a compiled extension, add concurrency, or fix the query. It is
also where you learn that the correct optimisation is very often to do less work
rather than to do the same work faster.
Django & FastAPI
The two dominant server frameworks, treated together because most backend Python
roles use one of them and the comparison is instructive. Django brings an ORM,
migrations, an admin interface, authentication and a strong set of conventions;
FastAPI brings type-driven validation, generated API documentation and
asynchronous handling built on ASGI. The subsection covers ORM behaviour and the
N+1 query problem, middleware, dependency injection, async endpoints and the
hazards of blocking inside them, and request validation. It exists separately
because framework behaviour is where backend Python interviews spend most of
their time, and because both frameworks have specific, well-known failure modes
an interviewer can reliably probe.
Testing in Python
pytest fixtures and their scopes, parametrisation, mocking and patching and
knowing where to point them, test doubles for external services, and reading a
coverage report honestly. Testing gets its own area in a dynamically typed
language for a reason that does not apply as strongly elsewhere: the compiler
catches almost nothing for you, so the test suite is carrying load that a type
checker carries in other ecosystems. The questions here are about boundaries —
what to patch, what to let run, when a test that mocks everything has stopped
proving anything.
Where it sits in a real system
In backend work Python occupies exactly the position Java or Node does: it
receives a request, applies logic, reads and writes a database, and emits events.
Django brings an ORM, an admin interface, authentication and migrations in one
opinionated package, which is why it dominates product teams that need a lot of
functioning software quickly. FastAPI takes the opposite approach — small, typed,
built on ASGI for asynchronous handling — and has become the default for APIs
whose main job is to sit in front of something else, very often a model.
In data work Python sits above the compute rather than doing it. This is the
single most important structural fact about the data stack and the one beginners
miss: pandas and NumPy are Python interfaces to array operations implemented in C
and Fortran. When you write a vectorised expression you are not running a Python
loop; you are asking a compiled library to run one for you over contiguous memory.
When you write an explicit for loop over rows, you are, and it is often a
hundred times slower. The same pattern repeats one level up — PySpark code
describes a job that a JVM executes, and PyTorch code describes a computation
graph that runs on a GPU. Python is the control plane.
Loading diagram…
The branch worth staring at is the first one: the syntax gives you almost no clue
which path you are on, and the performance difference between them is the whole
subject of the data-performance conversation.
In platform work Python is the language people reach for when a shell script has
become too complicated and a compiled tool is too much ceremony. Cloud provider
SDKs, Ansible, and a large share of internal deployment tooling live here.
How a Python service is actually served
One structural detail is missing from most tutorials and comes up constantly in
backend interviews: your framework does not talk to the network. Python defines
two interface conventions between a web server and an application. WSGI is
the older synchronous one, where the server calls your application, waits for it
to return, and hands back a response. ASGI is the asynchronous successor,
which additionally supports long-lived connections such as WebSockets and lets a
single worker interleave many in-flight requests. Django was built for WSGI and
has grown ASGI support; FastAPI is ASGI from the start, through Starlette.
In front of that interface sits a process manager. The conventional deployment
runs several worker processes — because of the interpreter lock, one process
cannot use several cores for Python work — with a supervisor restarting them and
distributing connections. That is why a Python service's capacity is usually
described in workers rather than threads, why a memory leak shows up as workers
being recycled rather than as a crash, and why the sizing question "how many
workers per container" is a real one with a real answer derived from cores and
from how much of each request is spent waiting.
Loading diagram…
The thing to notice is that the workers share nothing. Any in-process cache,
counter or rate limiter you write lives in exactly one worker, which is why
state that feels local has to move to Redis or the database the moment you scale
past a single process.
Who does this work
Backend engineers on Python teams live in Django or FastAPI, and their day
looks like any backend day: an endpoint, a migration, a slow query, an
integration. Data engineers build and operate pipelines, and their day is
largely orchestration and failure: a scheduled job that did not produce output, a
schema that changed upstream without warning, a backfill that must not
double-count. Data scientists frame questions and build models, working
mostly in notebooks and answering to a business stakeholder rather than an
on-call rota. Machine-learning engineers sit between those two and take the
data scientist's model into production, which means packaging, serving, latency,
monitoring and retraining. SREs and platform engineers write Python as the
glue around everything else.
Role
Typical output
Judged on
Backend engineer
An endpoint, a migration, a worker
Correctness and latency
Data engineer
A pipeline that runs nightly
Reliability and data quality
Data scientist
An analysis or a model
Insight and method
ML engineer
A served, monitored model
Latency, cost, retraining
Platform / SRE
Tooling and automation
Whether it runs unattended
The distinction that matters for interviews is who builds versus who explores.
Exploratory work is judged on the insight and the method; production work is
judged on whether it runs unattended at three in the morning. Candidates who
prepared for one and interviewed for the other are the most common mismatch in
this market.
Demand, adoption and how that is changing
Demand is very high and comes from two independent directions, which is unusual
and makes it robust. The first is that Python has become the default teaching
language and the default scripting language, so it accumulates users continuously
and every organisation ends up with Python somewhere. The second is that the
entire machine-learning ecosystem standardised on it. Model training, inference
libraries, evaluation tooling and the frameworks around large language models are
Python-first, and no credible competitor has emerged. That has pulled a large
volume of well-funded work into the language over the past decade.
Two shifts are worth knowing about. Performance has become a first-class concern
for CPython itself, with sustained interpreter optimisation work through the 3.11,
3.12 and 3.13 releases; and separately, a free-threaded build of CPython that can
run without the global interpreter lock became officially supported as an
experimental option in 3.13. Neither means the ecosystem's assumptions have
changed yet — most libraries still assume the lock exists — but the direction is
real. Meanwhile the tooling layer is being rewritten in Rust: Ruff for linting and
formatting, uv for packaging and environments, both markedly faster than what they
replace, and both adopted quickly enough that a candidate who has never heard of
them looks out of date.
A third shift is quieter and affects daily work more than either. Type
annotations have moved from a curiosity to an expectation in serious codebases.
They are not enforced by the interpreter, which is the point people miss, but a
checker in the pipeline turns them into a real constraint, and frameworks have
started deriving behaviour from them — FastAPI reads the annotations on a handler
to validate the request and generate the schema. The practical consequence is
that a Python interview in a backend role now frequently includes questions that
would have been unimaginable a decade ago: how you would model an
either-this-or-that value in the type system, what a generic type parameter buys
you, why a checker complains about something that plainly works.
The data half of the market has its own direction of travel, and it is towards
Python being the interface to something that is not Python. Query engines and
dataframe libraries written in Rust or C++ are increasingly exposed through
Python bindings, columnar in-memory formats let those libraries hand data to one
another without copying, and orchestration tools describe workflows in Python
that execute somewhere else entirely. None of this reduces the demand for Python
skill; it changes what the skill consists of, away from writing the loop and
towards knowing which engine to hand the work to and what the handover costs.
One honest caution about the demand. Because Python is easy to start with, the
entry-level end of the market is extremely crowded, and "I know Python" is close
to worthless as a differentiator. The demand is concentrated where the syntax is
the least of it: production data engineering, ML systems work, and backend
services at scale.
What makes it hard
The difficulty is not syntax, and this trips up people who conclude from an easy
start that the language is simple. The difficulty is that Python's runtime
semantics differ from the mental model the syntax suggests, and the divergence
only becomes visible when something breaks.
Names are bindings, not boxes. Assignment does not copy; it points another name at
the same object. Every mutable default, every aliased list, every surprising
mutation through a second reference follows from that, and no amount of careful
syntax avoids it:
defadd_item(item, basket=[]): # the list is created ONCE, when the function is defined
basket.append(item)
return basket
add_item("apple") # ['apple']
add_item("pear") # ['apple', 'pear'] -- same list, still there
The second hard thing is the global interpreter lock, and it is worth being
precise because it is widely misdescribed. The GIL is a lock inside CPython that
allows only one thread to execute Python bytecode at a time. So it does prevent
multiple threads speeding up CPU-bound pure-Python work: four threads doing
arithmetic will not beat one. It does not prevent concurrency in general.
Threads still help enormously for I/O-bound work, because the lock is released
while a thread waits on a socket or a disk. It does not apply to separate
processes, which is why multiprocessing sidesteps it entirely at the cost of
having to serialise data between processes. And it does not constrain NumPy or
similar libraries, which release the lock while running compiled code — one of
the reasons the scientific stack is fast despite the interpreter. A candidate who
says "Python can't do parallelism because of the GIL" has learned a slogan; the
correct answer names which of these four cases they are in.
The decision that follows is the one interviews actually ask about, and it has a
short answer if you frame it correctly.
Model
Helps with
Cost
Use when
Threads
Waiting on I/O
Shared mutable state, locking
Blocking libraries, modest concurrency
Processes
CPU-bound Python work
Serialisation, memory per process
Real computation in pure Python
asyncio
Very many concurrent waits
Whole call stack must cooperate
Thousands of sockets, async libraries
Compiled extension
CPU-bound numeric work
You must find or write one
Arrays, matrices, model inference
The row that catches people is asyncio's cost. An event loop only works if
nothing on it blocks, so a single synchronous database driver call inside a
coroutine stalls every other task in the process — the same failure the Node
ecosystem has, arriving in a language where most libraries are still synchronous
by default.
Loading diagram…
The third hard thing is the one nobody teaches, and it does more damage than the
other two combined: packaging and environments. Python's default behaviour is
to install into a shared location, so two projects wanting different versions of
the same library conflict, and system tools break when you upgrade something for
an unrelated reason. Virtual environments are the fix, and the fact that they are
a bolted-on convention rather than a language feature is why the landscape is
littered with competing solutions. Add native extensions — libraries with
compiled C, Fortran or CUDA components whose wheels must match your platform and
Python version — and you get the failures that consume real days: it works on my
machine, it works locally but not in the container, the same requirements file
resolved differently last Tuesday. Any senior Python engineer has strong opinions
here, formed painfully.
The fourth is memory, and it is subtler than the packaging problem because it
rarely announces itself. CPython frees an object when its reference count reaches
zero, which is immediate and predictable, and runs a separate cycle-detecting
collector for groups of objects that reference each other and would otherwise
never reach zero. Two consequences follow. Objects are expensive relative to
their contents — a list of a million small integers is not a million machine
words — which is why the data ecosystem uses typed arrays rather than lists. And
anything holding a reference keeps an object alive, so the classic Python leak is
not a leak at all but a cache, a module-level dictionary, or a closure that
captured more than you intended.
Why study it
Study Python if you want to work in data or machine learning, where there is
effectively no alternative — the entire ecosystem is here, and any other choice
means reimplementing what already exists. Study it if you want the fastest route
to being genuinely useful in a scripting or platform role, because the standard
library plus a cloud SDK covers an enormous amount of real automation. And study
it if you want optionality: Python is the most common second language in the
industry, so it is rarely wasted.
There is also an argument about the shape of the career it opens. Because Python
sits at the boundary between engineering and analysis, it is unusually easy to
move sideways from it — a backend engineer who picks up pandas can become useful
to a data team, and a data scientist who learns to package and serve becomes an
ML engineer. Few other languages sit on that many borders at once. If you do not
yet know which kind of work you want, that optionality is worth something real.
Do not study it expecting the ease of entry to convert into an easy job market at
the bottom. It will not; the junior end is saturated precisely because the
language is welcoming. Do not choose it if you specifically want to learn how
machines work — memory layout, ownership, cache behaviour — because Python is
designed to hide exactly that, and C, Rust or Go will teach it faster. And if
your target is high-throughput low-latency backend services, Python is a defensible
choice but not the obvious one, and you should be able to say why you picked it.
Your first hour
Skip the tutorial and go straight to the two things that separate people who know
the syntax from people who know the language.
python3 -m venv .venv # a project-local environmentsource .venv/bin/activate # on Windows: .venv\Scripts\activate
python -c "import sys; print(sys.executable)"# prove which interpreter you got
That third command is the point of the exercise: understand what changed when you
activated the environment, because environment confusion causes more wasted
Python hours than any language feature.
Then write one file that demonstrates the runtime to yourself. Define a function
with a mutable default argument and call it three times, printing the result each
time. Bind a list to two names, mutate through one, print the other. Write a
loop that sums a million integers in pure Python, time it with
time.perf_counter, then do the same sum with sum() over a range and compare.
Finally, run one CPU-bound function in four threads and then in four processes,
time both, and see which one actually got faster.
For the last ten minutes, do something that will save you more time than the rest
combined: profile something instead of guessing about it.
python -m cProfile -s cumtime your_script.py | head -20
Read the top of that output and find the line where your program is genuinely
spending its time. It is very unlikely to be where you expected. Doing this once,
early, establishes the habit that separates the people who optimise Python
effectively from the people who rewrite loops that were never the problem.
The artefact is a single annotated script whose output you can explain line by
line, plus one sentence in your own words saying what the GIL did to your thread
timing. If you can explain that script, you are ahead of most candidates who list
Python as a core skill.
What this is not
Python is not slow in the way the claim is usually meant. The interpreter is slow
relative to compiled languages, and a numerical workload written as Python loops
will be dreadful — but a numerical workload written properly spends its time in
compiled libraries, and most production Python is waiting on a database or a
network rather than on the interpreter. Naming the specific bottleneck is what
distinguishes a real answer from a repeated opinion.
Python is not untyped. It is dynamically typed with an optional annotation system
that has become standard practice in serious codebases; those annotations are not
enforced at runtime, which is exactly why a separate checker such as mypy or
Pyright exists in the pipeline. Notebooks are not a development environment for
production code, and shipping one is a recognisable signal. Data science is not
data engineering, and machine-learning engineering is a third thing again — the
titles are used loosely in adverts and precisely in interviews, so read the
responsibilities rather than the heading.
The GIL is also not a property of the Python language. It is an implementation
detail of CPython, which happens to be the implementation nearly everyone uses,
and the free-threaded work is an attempt to remove it without breaking the C
extensions that depend on the guarantees it quietly provided. Stating it as "the
GIL" without ever saying "CPython" is a small imprecision that tends to travel
with a shallow understanding of the rest.
And "I know Python" is not a claim that survives contact with a specific role. The
question that follows it is always which Python, and having a real answer — this
stack, this kind of system, these failure modes — is most of what separates a
strong candidate from a crowded field.
The three Python markets share a syntax and diverge on everything that gets
graded, so the most valuable preparation is deciding which one you are
interviewing for before you decide what to revise.