Users upload files up to 5GB and you must not proxy them through your API servers. How does the upload actually work?

Short answer

For 5GB uploads, your API should issue permission, not carry the bytes. Create a pending metadata row, return presigned multipart-upload URLs scoped to one object key, let the client send parts directly to object storage, then mark the row ready only after a storage event or server-side verification. Add cleanup for abandoned multipart uploads and quarantine/scanning before public reads.

What proxying five gigabytes costs you

Picture the version you are being asked to avoid. A 5GB request body arrives at an application server. That connection is now held open for minutes, occupying a worker, a socket and either memory or a temporary file. Your load balancer has an idle timeout that may fire mid-transfer. Your framework has a body-size limit you will raise, then raise again. A deploy that rolls that pod kills the upload at ninety per cent. And the same bytes cross the network twice, once to you and once to storage, so you pay for the transfer as well as the outage.

None of that is a tuning problem. The application server sits on the data path with nothing to contribute there. Take it off.

The API issues permission, not bandwidth

The shape is three parties and a clear division of labour. The client asks your API to begin an upload. The API writes a metadata row in a pending state, generates an object key, and returns a short-lived credential authorising exactly one operation on exactly that key. The client sends the bytes straight to object storage. Storage notifies your API when the object exists, and only that notification flips the row to ready.

sequenceDiagram
  participant C as Client
  participant A as API
  participant S as Object storage
  C->>A: begin upload with size and type
  A->>A: insert row state pending
  A-->>C: upload id plus signed URLs per part
  C->>S: PUT part 1..N in parallel
  C->>S: complete multipart upload
  S-->>A: object created notification
  A->>A: set row state ready

The interesting gap is between the last two arrows. Until that notification lands the object exists and your product does not know about it, which is the state every subsequent design decision has to survive.

Notice what the client never gets: your storage account credentials. A presigned URL carries a signature over a specific method, a specific key and an expiry time, so the holder can do that one thing until it lapses. Keep the expiry short. It only has to survive the start of the transfer, not its whole duration, because the signature is checked when the request begins.

Why five gigabytes has to arrive in parts

A single PUT of 5GB is one atomic operation with one outcome. Ninety-nine per cent uploaded, then a dropped connection, means zero per cent uploaded. On a domestic connection that is a real probability rather than a hypothetical. And it repeats.

Multipart upload changes the unit of failure. The client initiates an upload, sends the file as numbered parts, and completes it with the list of parts and their entity tags. Storage assembles the object server-side. A failed part is retried on its own, parts go in parallel so the transfer is limited by aggregate rather than single-stream throughput, and a client that stops can resume by asking which parts already exist. S3 documents the limits worth knowing: a single PUT tops out at 5GB, multipart goes to 5TB, and every part except the last must be at least 5MB. Your 5GB ceiling is exactly the point at which the single-PUT path stops being available anyway.

Choosing the part size is a small piece of real engineering. Larger parts mean fewer requests and more wasted bytes per retry; smaller parts mean the opposite. At 5GB with 100MB parts you have 50 parts, a retry costs 100MB, and the request count stays trivial. Say the number and say why, rather than saying "multipart".

The completion signal is where this design bruises

The client is not a trustworthy narrator. It can complete the upload and then lose its connection before telling you, be closed by the user, or lie. So the transition to ready must not depend on the client reporting success. Take it from the storage event instead. Or verify yourself that the object exists at the expected key with a plausible size.

Both channels can fail, so the pending row needs a sweeper. A job that walks rows pending for longer than some threshold, checks storage for the object, and either promotes or expires them closes the hole without adding a new failure mode. This is the part interviewers probe hardest, because it is where a design that sounded finished turns out to be missing its reconciliation.

Abandoned multipart uploads deserve their own sentence. Parts that were sent but never completed still occupy storage and still cost money, and they are invisible in an ordinary object listing. A bucket lifecycle rule that aborts incomplete multipart uploads after a few days is the fix, and it is the sort of operational detail that separates somebody who has run this from somebody who has read about it.

What the credential cannot enforce

A presigned URL scoped to a key and a method does not, on its own, constrain how many bytes the client sends. If the size matters, bind it at issue time where the mechanism allows it. A browser form-post policy can carry a content-length range condition; where you only have a presigned PUT, you verify size after the fact from the object metadata and reject the upload by refusing to mark it ready.

Content is the same story. You cannot inspect a file that has not arrived, and you must not trust the Content-Type the client declared. Land uploads in a quarantine location, verify the type by inspecting the bytes, scan if the product needs scanning, and only then copy or move the object to the location that serves reads. That layout also gives you somewhere to put the objects that fail, which a single-bucket design has no room for.

Your API's job in a large upload is to say who may write which key, and to be told afterwards what happened. Every byte on the wire between the client and storage is a byte you neither pay for twice nor hold a worker open for.

© 2026 Preptima. Originally published at preptima.com.