Frontend UX, Chunking & Progress Tracking: Engineering Guide

Uploading a 4 GB video from a flaky mobile connection is not a single HTTP request — it is a long-running, interruptible process that must survive tab refreshes, dropped Wi-Fi, expired tokens, and the user wandering out of cellular range. This guide treats the browser-side upload as three cooperating subsystems: a chunker that slices the file and feeds bytes through a bounded concurrency window, a progress channel that reconciles client byte counts with server-authoritative acknowledgements, and a retry-and-resume loop that turns transient failures into checkpoints instead of restarts. Get those three right and a transfer that loses its connection at 87% picks up at 87% — not at zero.

The patterns here assume you have already solved the storage contract: clients talk to object storage through S3 presigned URL workflows or a tus-style endpoint, and the backend follows direct-to-cloud upload patterns so that application servers never buffer whole files. Everything below lives on the client and in the thin control API in front of it: the chunk planner, the scheduler, the offset store, the progress reducer, and the error classifier. Roughly 900 lines of TypeScript in a mature implementation, and every one of them exists because a specific network reality broke a simpler version.

Architecture overview

A resilient upload is a loop, not a pipeline. The chunker emits fixed-size slices; each slice is uploaded under a concurrency limit; every success advances a durable offset; every failure is classified and either retried with backoff or surfaced as fatal. A separate progress channel reads the same offset state and pushes throttled updates to the UI, while a server-push channel reports backend processing progress that the client cannot observe directly.

Browser upload architecture: chunker, concurrency window, retry loop, progress channels A File is sliced by a chunker, dispatched through a concurrency window to object storage, with successes advancing a durable offset and failures routed into a retry and resume loop, while a progress channel and a server-push channel feed the UI. File / Blob selected input Chunker blob.slice() Concurrency window N slices in flight PUT Object storage part / byte range ETag Retry / resume classify + backoff requeue slice commit Durable offset IndexedDB record requeue Progress reducer → UI one write per frame Server push (SSE / WS) processing progress
The upload as a loop: the chunker feeds a bounded window, commits advance a durable offset, failures recirculate, and two independent channels feed one progress bar.

The single most important design decision is where truth lives. Client-side byte counters tell you what you sent, not what the server committed — and the gap between those two numbers is exactly the data you must not retransmit. Every robust upload keeps an authoritative offset (a byte position, a set of completed part numbers, or a tus Upload-Offset) and treats the network in between as untrusted.

The four places upload state lives

Debugging an upload means knowing which of four stores you are looking at, because they disagree constantly and each one lags the next.

  1. JavaScript heap. The File handle, the in-flight XMLHttpRequest objects, the per-chunk loaded counters. Lifetime: until navigation. This is the only store that knows about bytes currently on the wire, and it is the least trustworthy thing to render a completion state from.
  2. IndexedDB. The session record: file fingerprint, chunk size, uploadId, the committed part list, and the machine’s current state. Lifetime: until eviction or explicit delete. Writing here after every commit is what makes a reload survivable, and the schema details live in persisting upload state in IndexedDB.
  3. The edge. Cloudflare, an ALB, or nginx may buffer a request body before the origin sees it, which means upload.onprogress reaching 100% proves only that bytes reached a proxy 20 ms away, not that the origin accepted them. This is why a 5 GB “instant” upload on a fast link then hangs.
  4. Object storage. The only authority. A part exists once S3 returns an ETag, and only a ListParts or tus HEAD tells you the truth after a crash.

A rule that removes a whole family of bugs: never derive UI state from store 1 alone, and never resume from anything other than store 4. Store 2 is a cache of store 4 that lets you skip a round trip when it agrees.

Control plane and data plane

Split the client into a control plane that talks to your API in JSON and a data plane that moves bytes to storage. The control plane creates the session (POST /uploads{ uploadId, partSize, urls }), refreshes expiring signatures, and finalises (POST /uploads/:id/complete with the part list). The data plane does nothing but PUT slices at signed URLs and report ETags back. Keeping them separate means an expired credential is a control-plane refresh, not a failed upload, and it means the data plane can be swapped from S3 multipart to tus to a resumable GCS session URI without touching the UI. It also keeps your CORS surface small — only the storage origin needs the exotic headers described in fixing CORS preflight errors on S3 uploads.

Cross-cutting concerns

Three constraints shape every decision below: how long your credentials live, how many requests you are willing to pay for, and how much memory a phone will give you before the tab is killed.

Credential lifetime versus upload duration

A presigned URL has a fixed expiry. A 4 GB upload over a 3 Mbit/s uplink takes just over three hours. If you signed all 512 part URLs at session creation with a one-hour expiry, parts 200 onward will fail with 403 Forbidden and the body <Code>AccessDenied</Code><Message>Request has expired</Message> — and because that is a 403, a naive classifier marks it fatal and destroys three hours of work.

Two workable strategies. Either sign in batches — request the next 25 URLs from the control plane when the queue drops below 10 remaining — or issue one long-lived session token to your own endpoint and have the server sign per part on demand. Batch signing is cheaper (one API call per 25 parts) and keeps signature lifetime under 15 minutes, which is what a security review will ask for. Whichever you pick, treat 403 with a body containing Request has expired as retryable after refresh, distinct from a genuine authorisation failure. Rate limits on the signing endpoint matter too, since a client that re-signs on every retry can generate hundreds of calls per minute.

Request cost and the part-count budget

Part size is not a purely technical choice; it is a bill. S3 charges per PUT request (roughly $0.005 per 1,000 in us-east-1), caps multipart uploads at 10,000 parts, and rejects any non-final part below 5 MiB with EntityTooSmall. Meanwhile every failed part costs you a full retransmission of that part. Small parts mean more requests and more per-request overhead; large parts mean more wasted bytes per failure. The two curves cross in a band, and that band is where you should sit.

Part size trade-off: request count falls while retransmission cost rises For a 4 GiB upload, the number of PUT requests drops from 4096 at 1 MB parts to 64 at 64 MB parts, while the bytes re-sent after a single failed part rise from 1 MB to 64 MB; the curves cross between 8 and 16 MB. practical band Part size trade-off for one 4 GiB file 4096 820 512 256 64 1 MB 5 MB 8 MB 16 MB 64 MB 1 MB 5 MB 8 MB 16 MB 64 MB configured part size (log spaced) PUT requests issued MB re-sent per failed part
At 8–16 MB parts a 4 GiB upload costs 256–512 requests and wastes at most 16 MB per failure — the flat part of both curves.

For consumer video on mobile, 8 MB is a good default; for desktop uploads on wired connections, 16–32 MB reduces overhead without meaningful retry pain. The one hard constraint: fileSize / partSize must stay under 10,000, so any client that accepts files above about 80 GB has to scale part size with file size rather than hard-code it. The same arithmetic drives the decision covered in multipart vs single-PUT for files under 100 MB — below roughly 100 MB the machinery costs more than it saves.

Do not forget the cost of abandoned uploads. Parts that were never completed still occupy storage and still bill, invisibly, because they do not appear in a bucket listing. A lifecycle rule that aborts incomplete multipart uploads after 7 days is mandatory, not optional; see setting up S3 lifecycle rules for temporary uploads.

Memory, battery, and the mobile envelope

Blob.slice() does not copy — it returns a lazy view over the same underlying bytes, so planning 512 slices costs almost nothing. Memory is consumed when a slice is read: arrayBuffer() materialises the whole range, and four concurrent 16 MB reads plus browser send buffers can spike 150 MB. Mobile Safari kills tabs above roughly 200–300 MB. So: pass the Blob straight to xhr.send() and let the browser stream it rather than reading it yourself, and only materialise bytes when you genuinely need them — for checksums, for example, where hashing a 1 MB prefix beats hashing 4 GB. The mechanics of zero-copy slicing are in slicing large files with Blob.slice.

Concurrency has an energy cost too. Four parallel TLS connections on a cellular radio keep the modem in high-power state and drain battery measurably faster than two; on a congested link they also increase the loss rate, which increases retries, which increases transfer time. Three or four in flight is the practical ceiling for mobile; on desktop, six saturates most links and browsers cap per-origin HTTP/1.1 connections at six anyway.

The chunking layer: slicing, scheduling, and backpressure

The chunker has three jobs: produce a deterministic plan, keep exactly N slices in flight, and never lose a slice when one fails. Determinism matters because a resumed session must re-derive byte-identical ranges — if the second run uses a different part size, part 7 of run 2 is not part 7 of run 1 and the assembled object is corrupt. Persist the part size with the session and treat it as immutable for the life of that upload.

export interface ChunkPlan {
  index: number;      // 0-based; S3 part numbers are index + 1
  start: number;      // inclusive byte offset
  end: number;        // exclusive byte offset
}

/** Deterministic plan: same (size, chunkSize) always yields the same ranges. */
export function planChunks(size: number, chunkSize: number): ChunkPlan[] {
  if (chunkSize < 5 * 1024 * 1024) {
    throw new RangeError("S3 rejects non-final parts below 5 MiB (EntityTooSmall)");
  }
  const plans: ChunkPlan[] = [];
  for (let start = 0, index = 0; start < size; start += chunkSize, index += 1) {
    plans.push({ index, start, end: Math.min(start + chunkSize, size) });
  }
  return plans;
}

/**
 * Bounded-concurrency runner. `limit` lanes pull from a shared queue, so a slow
 * part never blocks the others: any free lane immediately takes the next plan.
 */
export async function runWindow(
  plans: readonly ChunkPlan[],
  limit: number,
  worker: (plan: ChunkPlan) => Promise<void>,
): Promise<void> {
  const queue = plans.slice();          // never mutate the caller's plan
  const failures: unknown[] = [];
  const laneCount = Math.max(1, Math.min(limit, queue.length));
  const lanes = Array.from({ length: laneCount }, async () => {
    for (let plan = queue.shift(); plan !== undefined; plan = queue.shift()) {
      try {
        await worker(plan);
      } catch (err) {
        failures.push(err);
        queue.length = 0;               // stop feeding lanes once we are dead
        return;
      }
    }
  });
  await Promise.all(lanes);
  if (failures.length > 0) throw failures[0];
}

The shared-queue shape is what makes the window self-balancing. A Promise.all over batches of four stalls the whole batch on its slowest member — on a lossy link that is the difference between 40 minutes and 70. Here, the moment a lane finishes it takes the next plan, so all four connections stay busy until the queue is empty.

Four-slot upload window over twelve parts with one retried failure A timeline showing four concurrent upload slots processing twelve parts; part seven fails with a 503 after 1.8 seconds, waits out a backoff interval, and is retried in the same slot while the other slots keep working. Bounded window: 12 parts, 4 slots, one 503 slot 1 slot 2 slot 3 slot 4 P1 P5 P9 P12 P2 P6 P10 P3 P7 503 backoff P7 retry P4 P8 P11 0 s 2 s 4 s 6 s 8 s 10 s 12 s Any free slot pulls the next queued part, so one slow or failed part never stalls the other three.
The retried part occupies its own slot while the rest of the window keeps draining the queue — total wall time grows by the backoff, not by the whole part.

Fingerprinting so a resumed session recognises the file

Resume requires proving that the file the user re-selected is the file you started. File.name is not enough (two phones both produce IMG_0001.HEIC), and hashing 4 GB to find out costs minutes of CPU. A prefix hash plus size plus lastModified is a good compromise: cheap, and wrong only if two files share the first megabyte, the exact byte length, and the modification millisecond.

/** Cheap, stable identity for a picked file: SHA-256 of the first MiB + size + mtime. */
export async function fingerprint(file: File): Promise<string> {
  const head = await file.slice(0, 1024 * 1024).arrayBuffer();
  const digest = await crypto.subtle.digest("SHA-256", head);
  const hex = Array.from(new Uint8Array(digest), (b) =>
    b.toString(16).padStart(2, "0"),
  ).join("");
  return `${hex.slice(0, 32)}-${file.size}-${file.lastModified}`;
}

crypto.subtle is only available in secure contexts, so this throws TypeError: Cannot read properties of undefined on plain HTTP — test on https:// or localhost. Full-file integrity hashing, and how to do it off the main thread, is covered in computing file checksums in the browser with Web Crypto.

Adaptive concurrency instead of a hard-coded four

A fixed window is tuned for one network. On hotel Wi-Fi it causes timeouts; on fibre it leaves throughput on the table. Additive-increase/multiplicative-decrease gives you a window that finds the link’s capacity in about ten parts and collapses instantly when the network degrades — the same control law TCP uses, applied at the request layer.

export class AdaptiveWindow {
  #limit: number;

  constructor(
    readonly min = 1,
    readonly max = 6,
    start = 3,
  ) {
    this.#limit = Math.min(Math.max(start, min), max);
  }

  get limit(): number {
    return this.#limit;
  }

  /** Widen only when parts complete comfortably inside the target latency. */
  onSuccess(elapsedMs: number, targetMs = 8_000): void {
    if (elapsedMs < targetMs && this.#limit < this.max) this.#limit += 1;
  }

  /** Halve on any failure or timeout — congestion, not bad luck. */
  onFailure(): void {
    this.#limit = Math.max(this.min, Math.floor(this.#limit / 2));
  }
}

Feed onFailure() from timeouts as well as HTTP errors; a part that takes longer than partSize / 50 kB/s is almost always a stalled connection rather than a slow one, and aborting it with an AbortController recovers faster than waiting. The abort mechanics are in aborting uploads with AbortController and timeouts.

Resumable upload state machines

Modelling an upload as an implicit set of booleans (isUploading, isPaused, hasError) produces impossible states — paused and uploading, errored and complete — and the bugs that follow. An explicit finite state machine makes illegal transitions unrepresentable: an upload is in exactly one of idle, uploading, paused, retrying, completed, or failed, and only declared edges move between them. The machine owns the durable offset and the resume handshake, so the rest of the UI just reads state.

export type UploadState =
  | "idle" | "uploading" | "paused" | "retrying" | "completed" | "failed";

export type UploadEvent =
  | { type: "START" } | { type: "PAUSE" } | { type: "RESUME" }
  | { type: "CHUNK_OK" } | { type: "ERROR"; fatal: boolean }
  | { type: "ALL_DONE" } | { type: "RETRY_NOW" };

const transitions: Record<UploadState, Partial<Record<UploadEvent["type"], UploadState>>> = {
  idle:      { START: "uploading" },
  uploading: { PAUSE: "paused", ERROR: "retrying", ALL_DONE: "completed", CHUNK_OK: "uploading" },
  paused:    { RESUME: "uploading" },
  retrying:  { RETRY_NOW: "uploading", ERROR: "failed", PAUSE: "paused" },
  completed: {},
  failed:    { START: "uploading" }, // allow a manual restart from the UI
};

export function nextState(current: UploadState, event: UploadEvent): UploadState {
  // A fatal ERROR always lands in `failed`, regardless of where it fired.
  if (event.type === "ERROR" && event.fatal) return "failed";
  const target = transitions[current][event.type];
  if (target === undefined) {
    console.warn(`Ignored ${event.type} in state ${current}`);
    return current; // no legal edge: stay put, never crash
  }
  return target;
}

Because the machine is data, you can render it, test every edge in isolation, and persist current alongside the offset. Two edges deserve attention. uploading → retrying must not clear the in-flight part list, or a recovered upload re-sends parts that were already committed. And completed has no outgoing edges on purpose: once the server has confirmed assembly, a stray late progress event from a lingering XMLHttpRequest cannot drag the UI backwards.

The resume handshake is where the machine earns its keep. On reload, before sending a byte, the machine moves idle → uploading only after reconciling: read the IndexedDB record, confirm the fingerprint matches the re-selected file, then ask storage what it actually holds (ListParts for S3 multipart, a HEAD returning Upload-Offset for tus). If the server reports fewer parts than the local record, trust the server and re-send the difference. The full reconciliation logic, including what to do when the server has garbage-collected the session, is in resumable upload state machines, and the protocol-level version is in building a resumable upload flow with tus.

Real-time upload progress events

Transfer progress and processing progress are different signals with different sources and different failure modes. Bytes leaving the browser are observable through XMLHttpRequest’s upload.onprogress — the Fetch API still cannot report request-body upload progress in Safari or Firefox, which is why XHR survives in 2026 for the data plane. Server-side work — virus scanning, transcoding, thumbnail generation — is invisible to the client and must be pushed back over Server-Sent Events or a WebSocket. The UI has to merge two clocks into one bar that never lies and never goes backwards.

export interface ChunkProgress { index: number; loaded: number; total: number; }

export function uploadChunkWithProgress(
  url: string,
  blob: Blob,
  index: number,
  onProgress: (p: ChunkProgress) => void,
  signal?: AbortSignal,
): Promise<string> {
  return new Promise((resolve, reject) => {
    const xhr = new XMLHttpRequest();
    xhr.open("PUT", url);
    xhr.upload.onprogress = (e) => {
      // lengthComputable is false when a proxy strips Content-Length.
      if (e.lengthComputable) onProgress({ index, loaded: e.loaded, total: e.total });
    };
    xhr.onload = () => {
      if (xhr.status >= 200 && xhr.status < 300) {
        // S3 returns the part ETag; you need it for CompleteMultipartUpload.
        resolve(xhr.getResponseHeader("ETag") ?? "");
      } else {
        reject(new Error(`HTTP ${xhr.status}: ${xhr.responseText.slice(0, 200)}`));
      }
    };
    xhr.onerror = () => reject(new Error("network error (CORS, DNS, or dropped socket)"));
    xhr.ontimeout = () => reject(new Error("chunk timeout"));
    xhr.timeout = 120_000;
    signal?.addEventListener("abort", () => xhr.abort(), { once: true });
    xhr.send(blob);
  });
}

Note the ETag read: it only works if the storage CORS policy lists ETag in ExposeHeaders. Omit it and getResponseHeader("ETag") silently returns null, CompleteMultipartUpload fails with InvalidPart, and the upload dies at 100%. That single line of bucket configuration causes more “it works locally” bug reports than anything else in this stack.

Aggregation is the second hard part. Four chunks in flight each report their own loaded; the bar needs their sum plus the bytes already committed, divided by the file size — and it needs to write that number to the DOM at most once per frame, because a gigabit link fires progress events faster than React can reconcile.

export class ProgressAggregator {
  #inflight = new Map<number, number>();
  #committed = 0;
  #frame = 0;

  constructor(
    private readonly total: number,
    private readonly render: (fraction: number) => void,
  ) {}

  report(index: number, loaded: number): void {
    this.#inflight.set(index, loaded);
    this.#schedule();
  }

  /** Called when the server confirms a part: move it from in-flight to committed. */
  commit(index: number, bytes: number): void {
    this.#inflight.delete(index);
    this.#committed += bytes;
    this.#schedule();
  }

  #schedule(): void {
    if (this.#frame !== 0) return;          // already queued for this frame
    this.#frame = requestAnimationFrame(() => {
      this.#frame = 0;
      let pending = 0;
      for (const loaded of this.#inflight.values()) pending += loaded;
      this.render(Math.min(1, (this.#committed + pending) / this.total));
    });
  }
}
Two progress clocks merged into one monotonic bar Byte-transfer events from XHR and processing events from Server-Sent Events are weighted, passed through a monotonic clamp, and rendered as a single progress bar where transfer occupies the first eighty-five percent. One bar the user trusts — it never decreases bytes on the wire · 0 → 85% 85 → 100 monotonic clamp max(last, weighted) xhr.upload.onprogress loaded / total, per slice SSE processing events scan → transcode → done × 0.85 × 0.15
Give transfer 85% of the bar and processing the last 15%: the bar keeps moving during server-side work instead of freezing at 99%.
const TRANSFER_SHARE = 0.85;

/** Merge the two clocks; `last` is the previously rendered fraction. */
export function combineClocks(transfer: number, processing: number, last: number): number {
  const raw = transfer * TRANSFER_SHARE + processing * (1 - TRANSFER_SHARE);
  return Math.max(last, Math.min(1, raw));  // clamp: never rewind, never exceed 1
}

Choosing between the two server-push transports is a question of direction, not fashion — the comparison is worked through in WebSockets vs SSE for upload progress, and the throttling, lengthComputable fallbacks and reconnect handling are in real-time upload progress events.

Upload error recovery patterns

The default behaviour of fetch on failure is to throw and forget. A resilient client instead classifies the failure, decides whether it is retryable, and — if so — waits a jittered, exponentially growing interval before retrying the same slice against the same offset. Retries must be idempotent: re-PUTting part 7 to the same byte range produces the same object, so duplicate delivery is harmless. The browser’s online/offline events let you pause the whole machine when the network drops instead of burning the retry budget against a dead link.

const FATAL_STATUS = new Set([400, 401, 404, 405, 413, 422]);

export function isRetryable(status: number | null, body = ""): boolean {
  if (status === null) return true;                      // transport error, no response
  if (status === 403 && body.includes("Request has expired")) return true; // re-sign, then retry
  if (FATAL_STATUS.has(status)) return false;            // client, auth, validation
  return status >= 500 || status === 408 || status === 429;
}

/** Full jitter (AWS "Exponential Backoff and Jitter"): sleep in [0, min(cap, base·2^n)). */
export function backoffDelay(attempt: number, baseMs = 500, capMs = 30_000): number {
  const window = Math.min(capMs, baseMs * 2 ** attempt);
  return Math.floor(Math.random() * window);
}

export async function withRetry<T>(
  task: () => Promise<T>,
  retryable: (err: unknown) => boolean,
  maxAttempts = 6,
): Promise<T> {
  for (let attempt = 1; ; attempt += 1) {
    try {
      return await task();
    } catch (err) {
      if (attempt >= maxAttempts || !retryable(err)) throw err;
      if (!navigator.onLine) {
        await new Promise<void>((resolve) =>
          globalThis.addEventListener("online", () => resolve(), { once: true }),
        );
      }
      await new Promise((resolve) => setTimeout(resolve, backoffDelay(attempt)));
    }
  }
}

Full jitter rather than “exponential plus a little noise” is deliberate. When a storage endpoint returns 503 SlowDown to every in-flight part at once, four retries scheduled at exactly 1,000 ms hit it simultaneously and reproduce the overload; sleeping a uniform random interval in [0, 1000) spreads them. The measurements behind that choice are in implementing exponential backoff for failed chunks.

Waiting on the online event rather than polling matters on mobile, where a tunnel or a lift can black out the radio for minutes. navigator.onLine is famously optimistic — it reports true for a captive portal that answers every request with a login page — so treat it as a cheap negative signal only: false definitely means offline, true means “worth trying”. Recovery after a genuine disconnection, including re-handshaking a stale offset, is in resuming uploads after network loss, and the idempotency-key pattern that makes control-plane retries safe is in retrying fetch uploads with idempotency keys.

Client-side media preprocessing

The cheapest chunk is the one you never send. A 12-megapixel phone photo is 4–6 MB as HEIC and 12 MB as JPEG; downscaled to 2048 px on the long edge and re-encoded as WebP it is around 400 KB with no visible loss at display sizes. For a gallery upload of 40 photos that turns a 4-minute transfer into 20 seconds, removes the server-side derivative job entirely, and strips GPS coordinates that you probably should not have received in the first place.

/** Downscale and re-encode in the browser. Returns a Blob ready for the chunker. */
export async function downscaleImage(
  file: File,
  maxEdge = 2048,
  quality = 0.82,
): Promise<Blob> {
  const bitmap = await createImageBitmap(file, { imageOrientation: "from-image" });
  const scale = Math.min(1, maxEdge / Math.max(bitmap.width, bitmap.height));
  const width = Math.round(bitmap.width * scale);
  const height = Math.round(bitmap.height * scale);
  const canvas = new OffscreenCanvas(width, height);
  const ctx = canvas.getContext("2d");
  if (ctx === null) {
    bitmap.close();
    throw new Error("2d context unavailable — fall back to uploading the original");
  }
  ctx.drawImage(bitmap, 0, 0, width, height);
  bitmap.close();                    // release the decoded surface immediately
  return canvas.convertToBlob({ type: "image/webp", quality });
}

Two cautions. imageOrientation: "from-image" bakes the EXIF rotation into the pixels — without it, portrait photos from iOS arrive sideways, because you have discarded the orientation tag along with the rest of the metadata. And re-encoding is lossy and irreversible: for anything a user may later download as an original (RAW files, design assets, legal documents) preprocess a derivative for preview and upload the original untouched. OffscreenCanvas runs happily inside a Web Worker, which keeps a 40-photo batch from freezing the main thread; the full pattern, including the WebCodecs route for video, is in client-side media preprocessing.

Configuration reference

Every one of these belongs in a config object that is persisted with the session, not scattered as literals through the code — a resumed upload must reconstruct the exact same plan.

Option Type Default Effect
partSize number (bytes) 8 * 1024 * 1024 Slice length. Must be ≥ 5 MiB for S3 multipart and give ≤ 10,000 parts. Immutable per session.
concurrency number 4 Slices in flight. 3–4 on mobile, up to 6 on desktop; browsers cap HTTP/1.1 at 6 per origin.
adaptive boolean true Enables AIMD resizing of the window between minConcurrency and concurrency.
maxAttempts number 6 Attempts per slice before the machine goes failed. With a 30 s cap this is ~1 minute of retrying.
baseDelayMs number 500 First backoff window. Full jitter samples uniformly from [0, window).
capDelayMs number 30000 Upper bound on the backoff window; beyond this users assume the app is broken.
chunkTimeoutMs number 120000 xhr.timeout per slice. Should exceed partSize / 50 kB/s or slow links time out spuriously.
fingerprintBytes number 1048576 Prefix hashed for file identity. Larger is safer, slower; 1 MiB hashes in about 6 ms.
transferShare number 0.85 Fraction of the progress bar owned by byte transfer; the rest is server-side processing.
progressWrites "raf" | "interval" "raf" raf coalesces to one DOM write per frame; interval (250 ms) is better for background tabs.
signBatchSize number 25 Presigned URLs fetched per control-plane call. Keeps signature TTL short without chatty signing.
sessionTtlHours number 168 How long the IndexedDB record is offered for resume. Match the bucket’s abort-incomplete rule.

Decision matrix

Choose the transport and progress strategy by file size, network volatility, and whether the backend does post-upload processing.

Scenario Transport Progress channel Resume strategy Concurrency
Small file (< 5 MB), stable network Single XHR PUT upload.onprogress Restart on failure 1
Large file, lossy mobile network Chunked multipart / tus Aggregated upload.onprogress Durable offset + ListParts 3–4
Long server-side processing Chunked + control channel SSE for the processing clock Durable offset 3–4
Server needs to pause or cancel Chunked + WebSocket WebSocket frames Durable offset + Upload-Offset 4
Cross-session, cross-device resume tus protocol HEAD poll then SSE Upload-Offset + IndexedDB 2–4
Many small images (gallery) Preprocess, then single PUT each Count of completed files Per-file retry, no offsets 4–6

Verification

An upload client that has never been tested against a hostile network is untested. These four checks catch the majority of regressions before users do.

Prove the planner and the window are correct. The scheduler is pure logic, so test it in Node without a browser:

import { strict as assert } from "node:assert";
import { planChunks, runWindow } from "./chunker.js";

const FOUR_GIB = 4 * 1024 * 1024 * 1024;
const plans = planChunks(FOUR_GIB, 8 * 1024 * 1024);
assert.equal(plans.length, 512);
assert.equal(plans[0].start, 0);
assert.equal(plans[511].end, FOUR_GIB);
assert.equal(plans.reduce((sum, p) => sum + (p.end - p.start), 0), FOUR_GIB);

let active = 0;
let peak = 0;
await runWindow(plans.slice(0, 32), 4, async () => {
  active += 1;
  peak = Math.max(peak, active);
  await new Promise((resolve) => setTimeout(resolve, 5));
  active -= 1;
});
assert.equal(peak, 4);
console.log("planner + window OK: 512 parts, window never exceeded 4");

Prove resume works. Start a real upload, kill it at roughly 40% (DevTools → Network → Offline), reload the tab, re-select the same file, and confirm the client asks storage for the committed offset before sending anything. For tus that is a single request:

curl -s -I -X HEAD https://uploads.example.com/files/9f2c4a7e \
  -H "Tus-Resumable: 1.0.0"
# HTTP/2 200
# Tus-Resumable: 1.0.0
# Upload-Offset: 41943040
# Upload-Length: 104857600

The resumed run must issue its first PATCH/PUT at byte 41943040, not byte 0. Watch the Network panel: if you see a request for part 1, your reconciliation is broken.

Prove the bar is honest. Throttle to “Slow 3G” and record the DOM writes. The fraction must be monotonically non-decreasing, must reach 0.85 only when the last part is committed, and must reach 1.0 only after the processing channel says done. A quick assertion during development:

let last = 0;
export function assertMonotonic(fraction: number): void {
  if (fraction < last - 1e-9) {
    throw new Error(`progress went backwards: ${last.toFixed(4)}${fraction.toFixed(4)}`);
  }
  last = fraction;
}

Prove the object is intact. After completion, compare the storage-side checksum with a locally computed one. For a multipart object the S3 ETag is a hash of hashes with a -512 suffix, so it will not equal the file’s MD5 — either compute the same composite yourself or, better, ask the bucket for a SHA256 checksum on completion and compare that. A mismatch means slices were assembled out of order or a retry duplicated a range.

Common failure modes

  • Stalled progress at 99%. The transfer finished but the bar waits on a confirmation that never arrives. Root cause: the client marks “complete” on the last 200 instead of on a server-authoritative completion event. Fix: enter completed only when the assembly channel reports done, and give processing its own visible share of the bar.
  • InvalidPart on completion. CompleteMultipartUpload fails with One or more of the specified parts could not be found. Root cause: missing ExposeHeaders: ["ETag"] in the bucket CORS policy, so every collected ETag was null. Fix: expose the header and re-run; there is no client-side workaround.
  • EntityTooSmall. Your proposed upload is smaller than the minimum allowed size on part 1 of 3. Root cause: a part size below 5 MiB, usually from someone “tuning for mobile”. Fix: 5 MiB floor for every part except the last, enforced in planChunks.
  • net::ERR_UPLOAD_FILE_CHANGED. Chrome aborts mid-upload with this after the user edits or re-exports the file while it is being sent; Safari surfaces the same condition as NotReadableError. Fix: catch it, invalidate the session, re-fingerprint, and ask the user to re-select — the object on disk is no longer the object you planned.
  • Duplicate or reordered slices. The assembled file is corrupt at a part boundary. Root cause: the server keyed chunks by arrival order rather than by part number or byte range. Fix: key every write by (uploadId, partNumber) so re-delivery overwrites rather than appends, then verify the final checksum.
  • Retry storms after a brief 503. A storage endpoint hiccups, every in-flight part retries on the same schedule, and the endpoint stays down. Root cause: synchronised, un-jittered backoff. Fix: full jitter plus an online gate, and halve the concurrency window on failure.
  • 413 Request Entity Too Large from a proxy you forgot about. Only appears in production, because the dev server has no nginx in front. Root cause: client_max_body_size 1m or a Cloudflare plan limit sitting between the browser and the origin. Fix: raise the limit or move to direct-to-storage uploads — see raising Nginx and Cloudflare upload size limits, and treat the client side as described in handling 413 and 507 errors during uploads.
  • Frozen UI on a fast link. A gigabit connection fires thousands of progress events and the framework re-renders on each. Root cause: unthrottled state writes. Fix: coalesce to one requestAnimationFrame write per frame, as in ProgressAggregator above.
  • QuotaExceededError writing the session record. Private browsing modes give IndexedDB a few megabytes. Root cause: storing chunk payloads rather than offsets. Fix: persist only metadata — a session record should be well under 1 KB.

Frequently Asked Questions

Should I use the Fetch API or XMLHttpRequest for uploads?

Use fetch for the control plane — creating sessions, refreshing signatures, completing uploads — and XMLHttpRequest for the bytes whenever you need a progress bar, because fetch still cannot report request-body progress outside Chromium. ReadableStream request bodies are the eventual replacement and are worth prototyping today via uploading with ReadableStream request bodies, but they require HTTP/2, a duplex: "half" flag, and a fallback path.

Do I need a Web Worker for slicing and hashing?

Slicing, no — Blob.slice() is a zero-copy view and takes microseconds. Hashing, yes, once you go past a megabyte or two: crypto.subtle.digest over 100 MB blocks the main thread for hundreds of milliseconds and janks every animation on the page. Move hashing and any canvas re-encoding into a worker and keep the scheduler on the main thread where it can see online/offline and visibility changes.

How do I know the user re-selected the same file after a crash?

Compare a fingerprint, not a filename. Size plus lastModified plus a hash of the first megabyte is fast and specific enough in practice. Be aware that lastModified can differ across devices for the same logical file, so a fingerprint mismatch should prompt “start a new upload?” rather than a hard error.

Is tus worth adopting instead of a bespoke chunker?

If you control both ends and only ever target one storage backend, direct multipart against presigned URLs is less machinery. Choose tus when you need cross-device resume, a well-specified offset handshake, or existing client libraries for iOS and Android — the trade-off is that bytes now flow through your server or a tus proxy rather than straight to storage.

Should the progress bar show a time estimate?

Only if you smooth it. A naive remainingBytes / instantaneousRate swings from “2 minutes” to “4 hours” on every radio hiccup and destroys trust faster than showing nothing. Use an exponentially weighted average of throughput over the last 10–20 seconds, round aggressively (“about 3 minutes”, not “2:47”), and never render a countdown that goes up — hold the previous estimate until the smoothed rate recovers. The same reducer that owns the progress clocks is the right place to keep that average, since it already sees every committed byte.