Blockchain and Web3 Engineering

What this area actually covers

Two things that people bundle into one word and that require different skills.

The first is writing programs for a replicated deterministic machine. A smart contract is code deployed at an address on a public network, where the bytecode is effectively permanent, the storage persists indefinitely, every execution is re-run identically by thousands of independent machines, and every byte of state is visible to everyone forever. Those constraints are the whole discipline. You cannot patch, you cannot log privately, you cannot call an API, you cannot schedule a job, you cannot assume a caller is a human, and every resource you consume is billed to whoever invoked you. Programming under those rules is less like writing a service and more like writing firmware for a device you will never be able to recall — except that the device is also holding money and anyone in the world may prod it.

The second is the ordinary software that sits around those contracts and makes them usable. Wallets that hold keys and construct transactions. Backends that read chain state through remote procedure calls to nodes they do not operate. Indexers that turn a stream of events into queryable tables. Storage layers for the files that are too large or too mutable to live on chain. This half is closer to conventional distributed-systems work, with the twist that your database is a global replicated log operated by strangers, its view of the present is provisional, and the writes cost money and cannot be rolled back.

It is worth naming what gets wrongly bundled in. Trading strategy and market-making are finance, not blockchain engineering, even though they employ a lot of blockchain engineers. Cryptography research is a separate field, and almost nobody in this area implements a primitive; they use signature verification and hashing as given. Consensus protocol development — writing the client software itself — is a small, specialised world of a few hundred people, and it is not what the job description usually means. And token economics, the design of incentives and supply schedules, is a discipline of its own that engineers are frequently asked to opine on and rarely trained in.

The honest boundary of the area is this: everything from "how do I keep this contract from being drained" up to "how does my web application read chain state reliably", and nothing above or below it.

The two areas underneath

This section splits in two because the two halves fail differently and are interviewed differently. One is examined like a security review; the other is examined like a systems design round.

Smart Contracts covers Solidity as a language, the EVM's execution and cost model, the vulnerability classes by name, and the patterns that structure a contract safely. It exists as its own subsection because the reasoning is adversarial and local: the questions are about a specific function, a specific ordering of statements, a specific assumption that turns out to be false when the caller is a contract rather than a person. Open it expecting re-entrancy and the checks-effects-interactions ordering that removes it, storage layout and why the slot you touch is the unit of cost, and proxy upgradeability with the trust it quietly transfers from the code to whoever holds a key. The register is close to code review — you are being asked to read a function and say what is wrong with it, then say what invariant you would preserve instead.

Web3 Architecture covers the surrounding system: what a wallet actually does when a user clicks confirm, how nonces sequence an account's transactions and what a stuck one blocks, how to depend on a hosted RPC provider without inheriting its rate limits and its lagging view, and when content-addressed storage is genuinely the right answer rather than a database. It is separate because none of it is a contract problem and all of it is where production incidents come from. A perfectly-audited contract still breaks its application if the backend books a transaction hash as a completed payment, if two workers sign against the same nonce, or if the metadata gateway everyone depends on goes down. Open it expecting failure modes, confirmation policies, reorganisation handling, and the reasons a second RPC provider is not automatically a fallback.

If you have to choose, choose by the job. A protocol team interviews mostly from the first; a product team building on existing protocols interviews mostly from the second, and is often relieved to find a candidate who takes the boring reliability questions seriously.

Where it sits in a real system

A production application in this space is a conventional web stack with an unusual dependency in the middle of it. The chain is not the application; it is the settlement layer that the application defers to for the small number of facts that must be shared and verifiable.

flowchart TD
    U[Browser client] --> W[Wallet holds keys and signs]
    U --> B[Your backend and API]
    W -->|signed transaction| R[RPC node provider]
    B -->|reads| R
    R --> C[Chain state and contracts]
    C -->|events| I[Indexer and your database]
    I --> B
    B --> S[Content-addressed store for large files]

The arrow to notice is the one from the wallet straight past your backend. Your application does not hold the user's key and cannot act for them, so the write path leaves your control entirely at the point of signing — you can propose a transaction and observe the result, and that is all. Most of the architectural difficulty in the diagram comes from that single fact, because it means your system's state is derived from something you neither own nor can correct.

The read path has the mirror-image problem. The indexer exists because the chain is a poor query engine: it answers by identifier and by block, so any question with a filter, an ordering or a page in it has to be answered from your own store. That store is a projection of an append-only log that can occasionally retract its most recent entries, so the projection has to be reversible.

Money moves in that diagram, so it is worth walking a single write end to end. A user asks to withdraw. Your application constructs the call and hands it to the wallet, which fills in the account's next nonce and the fee it is willing to pay, hashes the encoded transaction and signs the hash locally. The signed bytes go to a node, which validates them cheaply and gossips them to peers. A block proposer eventually selects the transaction, executes it against the current state, and includes it. Your backend fetches the receipt, checks its status field — because a reverted transaction has a receipt too, and cost the sender gas — and only then decides whether the effect is real, based on how deep in the chain the block now is.

sequenceDiagram
    participant U as User and wallet
    participant B as Your backend
    participant N as RPC node
    participant P as Block proposer
    U->>B: request a withdrawal
    B->>U: unsigned call data
    U->>N: signed transaction
    N->>P: gossip to mempool
    P->>N: block including it
    B->>N: fetch receipt and check status
    B->>B: wait for depth before crediting

The gap between the third and fifth arrows is the one that generates support tickets. Between them the transaction exists, has an identifier, and has no outcome — and it may never acquire one. Treating that identifier as a receipt is the single most common architectural mistake in the area.

The mechanisms you have to hold in your head

Four ideas do most of the work, and someone who has these can reason about most of the rest from first principles.

Determinism, and what it forbids. Every node must compute the same result from the same inputs, forever. That is why a contract cannot make an HTTP request, cannot read a clock more precise than a block timestamp, and has no source of randomness that is both unpredictable and verifiable. Every one of those needs an external party to push the information in, which is what an oracle is — and an oracle is a trust boundary, not a library.

Gas, and what it is really pricing. Gas meters work so that no transaction runs forever and so that resources are paid for in proportion to their cost. The prices are not uniform: computation is transient and cheap, memory lasts one call and is cheap, and persistent storage is expensive because every validating machine must carry it indefinitely. Almost all useful optimisation follows from that one asymmetry, and it explains a great deal of contract design that otherwise looks perverse — why history goes in events rather than arrays, why an allowlist becomes a merkle root, why a distribution is a pull rather than a push.

Control transfer. An external call hands execution to code you did not write, which may call back into you before your function has finished. This is the root of re-entrancy and of its modern variants through token hooks and callbacks, and the defence is an ordering discipline rather than a library: every invariant your contract depends on must be true before control leaves it.

Provisional finality. The most recent blocks are not settled. A reorganisation can discard a block along with every transaction that appeared only in it, so anything derived from recent chain state must be reversible. Chains differ in how they express this: some give you an explicit finalised checkpoint you can read, and on others depth is a probabilistic argument about the cost of a rewrite. Either way, your application chooses a threshold, and the choice is a risk judgement scaled to the value at stake.

Who does this work

The titles overlap and the work does not, so it is worth reading a job description for which of these it actually describes.

A smart contract engineer's day is unusually slow by conventional standards, and that is the point rather than a complaint. A change of a few dozen lines can involve a written specification of the invariant, property-based tests that try to violate it, a review by someone who is trying to break it, and a deployment plan with a staged rollout — because there is no hotfix. Engineers arriving from web development often find this the hardest adjustment: the ratio of thinking to typing is inverted.

The infrastructure and full-stack roles look much more familiar. Queues, caches, cursors, retries, monitoring, an on-call rotation. The differences are that one of your dependencies is operated by a third party with rate limits and a lagging view, that some of your writes cost money and cannot be undone, and that your users hold their own credentials so you cannot fix their mistakes.

There are also the roles adjacent to engineering that get confused with it. Auditing firms employ engineers full-time to review other people's code, which is a genuinely different job with a different skill curve. Node operators and staking providers do infrastructure work at a scale most product teams never touch. And there is a substantial population of people whose title says blockchain and whose work is integration, reporting and reconciliation against a chain someone else designed — perfectly real work, and not what a protocol interview tests.

Demand, adoption and how that is changing

Be clear-eyed about this, because it is the question the page owes you an honest answer to.

Demand for blockchain engineers is real, specialised, and much smaller than the industry's noise level suggests. It is also unusually cyclical. Hiring in this space tracks asset prices and funding availability with very little lag, which means the same skill set can be in visible shortage and then visibly surplus within a couple of years, and that pattern has repeated more than once. If you are choosing a specialisation for job security alone, this is not the one, and anybody telling you otherwise is selling something.

Underneath the cycle there are a few durable pockets. Contract security review is the most consistent, because the value at risk makes review worth paying for regardless of sentiment, and because good reviewers are genuinely scarce — the skill takes years and does not transfer easily from general application security. Infrastructure is the second, because nodes, indexers and RPC providers are load-bearing for everyone else and consolidate into a smaller number of larger operators rather than disappearing. Institutional and regulated work is the third and the least glamorous: custody, tokenised instruments, settlement and compliance reporting, where the driver is a regulatory or operational mandate rather than a retail market.

The area is also genuinely contested, and an interview may well include a sceptical question about whether any of it is useful. Take that seriously rather than defensively. A large share of activity is speculative, the user experience remains difficult, and many deployed systems would be simpler, faster and cheaper as an ordinary database with an audit log. The property that is actually hard to obtain elsewhere is narrow: verifiable shared state between parties who will not appoint a common operator. Where that is the requirement, the machinery earns its cost. Where it is not, it is an expensive way to get a slow database. An engineer who can draw that line is more trustworthy on design decisions than one who cannot, and interviewers treat it as a signal rather than as disloyalty.

Two structural changes are worth knowing about because they change what the work looks like rather than how much of it there is. Execution has been moving off the main settlement layers onto networks that batch and post back to them, which lowers the cost of a user action and adds a set of cross-network concerns — deployment per network, bridging assumptions, and withdrawal delays. And account handling has been moving from a single-signature account towards programmable accounts, with the consequence that "who is authorised" becomes contract logic rather than a protocol rule. Both increase the surface a full-stack engineer must understand.

What makes it hard

Not the language. Solidity is a small language and an experienced engineer can read it in an afternoon. The difficulty is elsewhere, in four places.

Every input is hostile and every caller may be a program. Conventional server code assumes a distribution of ordinary users with a minority of attackers. Here the profitable attack is found by anyone reading your published bytecode, executed by a contract that composes several of your functions in an order you never considered, and financed with capital borrowed and repaid inside the same transaction. Reasoning about that requires holding invariants rather than test cases in your head, and it is the skill that experience is least substitutable for.

Mistakes are irreversible and public. There is no rollback, no rotation of a leaked value out of history, and no quiet fix. A deployed bug is visible to everyone who might exploit it at the same moment you notice it, which inverts the usual incident response: disclosing that you are patching can be the thing that gets you exploited. This is why the discipline around deployment is heavier than the code volume seems to justify.

Correctness includes economics. A contract can be free of memory bugs, arithmetic bugs and access-control bugs, and still be broken by a participant who follows the rules exactly and profits at everyone else's expense. Manipulating a price source your contract trusts, or racing a reward everyone can see, are not violations of your code — they are consequences of your design. Software engineering training does not cover this, and it is where the most expensive failures live.

The environment is provisional and someone else's. Recent state can be withdrawn, your view of the present comes from a node you do not operate, your transaction's position in a block is a stranger's decision, and the thing your user must confirm is rendered by software you do not control. Building reliably means treating each of those as a failure domain with its own handling, which is a lot of machinery for what a newcomer expects to be a database write.

To that, add a cultural difficulty worth naming: the signal-to-noise ratio of available material is poor. A great deal of writing in this space is promotional, out of date, or subtly wrong, and version-sensitive details change with protocol upgrades. Learning here requires more discrimination about sources than most areas, and the primary documentation and the improvement proposals themselves are usually the fastest route.

Why study it

Three honest reasons, and one clear case for skipping it.

The first reason is that the constraints teach you things that transfer. Working where you cannot patch, cannot log privately and cannot trust any caller produces habits — stating invariants, ordering state changes before external calls, designing for irreversibility, treating every dependency's view as provisional — that make you better at ordinary distributed systems. Engineers come back from this area noticeably more careful about idempotency and failure ordering, and that is not a coincidence.

The second is that the security specialism is unusually meritocratic and genuinely scarce. The artefacts are public, so a body of good public review work is verifiable in a way that most engineering output is not, and the ceiling is high because the value at risk is high. It is a slow build and it does not require anyone's permission to start.

The third is narrow and real: if the problem in front of you actually requires verifiable shared state between parties who will not appoint a common operator, there is no adjacent technology that provides it, and the people who can build it competently are few.

Who should skip it. If you want the fastest route to a well-paid, stable engineering job, this is not it — backend, data or platform engineering are larger, steadier markets with more employers per city. If you are drawn by asset prices rather than by the engineering, the cycle will make that decision for you. And if you dislike work where a single mistake is public and permanent, this environment will be a poor fit no matter how interesting the problems are, because that pressure is not a phase of the job, it is the job.

Your first hour

Do not start with a tutorial that deploys a token. Start by writing the vulnerable contract, exploiting it, and then fixing it, because that loop teaches the thing the area is actually about and it fits in an hour.

Install a contract development toolchain by following its own documentation — Foundry and Hardhat are both fine, and Foundry has the advantage that the tests are written in Solidity, so you stay in one language. Then create a project and write two contracts: a vault with a deposit and a withdrawal, and an attacker.

// Vault.sol - deliberately wrong. The state update comes after the call out.
contract Vault {
    mapping(address => uint256) public balances;

    function deposit() external payable {
        balances[msg.sender] += msg.value;
    }

    function withdraw() external {
        uint256 amount = balances[msg.sender];
        require(amount > 0, "nothing to withdraw");
        (bool ok, ) = msg.sender.call{value: amount}("");
        require(ok, "send failed");
        balances[msg.sender] = 0;   // move this line up to fix it
    }
}

Write an attacker contract whose receive() calls withdraw() again while it still has a balance recorded, deposit a small amount from it, and run a test asserting that the vault ends up empty. The test should pass, which is the uncomfortable part. Then move the assignment above the external call, run the same test, and watch it fail because the second withdrawal now reverts.

That is the artefact for the hour: a repository with a failing exploit test against fixed code, and a passing one against the broken version. It is worth more in an interview than a deployed token, because you can explain the invariant you preserved rather than the steps you followed.

If you have time left, add two things. Print the gas used by your withdrawal, add a second storage variable, and print it again, so the cost of a slot stops being an abstraction. And write a three-line script that fetches a receipt for any transaction hash on a public network and prints its status field, so the difference between "included" and "succeeded" is something you have seen rather than read.

What this is not

It is not cryptography. You will use hashes and signature recovery as primitives and you will almost certainly never implement one. A cryptography interview and a smart contract interview have nearly nothing in common.

It is not trading, and it is not finance. Understanding a lending market well enough to build one is useful; the strategy and risk work is a different profession that happens to share an industry.

It is not consensus protocol engineering. Writing the client software that implements proof of stake is a small specialism with its own hiring pipeline, and a job advertised as blockchain engineering essentially never means it. You are expected to know what consensus decides and what finality means, not to implement either.

It is not a distributed database. This is the misconception that costs companies the most money, because the machinery looks superficially like replication and its properties are almost the opposite: writes are slow and expensive, everything is public, queries are not supported, and the design goal is to remove the operator rather than to scale one. If a single organisation can legitimately operate the system, a database and an append-only audit log will beat a chain on every axis that matters.

And it is not a way to avoid ordinary software engineering. The contract is usually the smallest component in the system. Everything around it — the keys, the nonces, the confirmations, the indexer, the reorganisation handling, the storage, the monitoring — is conventional engineering done under unusual constraints, and that is where most of the work and most of the incidents actually are.

The chain is a settlement layer for the few facts that must be shared and verifiable. Almost everything else in a working system is ordinary software, and treating it as ordinary software done carefully is what separates the engineers who ship from the ones who deploy.