Server-Side File Validation #

Every fact a browser sends you about an uploaded file — the filename, the extension, the Content-Type header — is a string the client chose, and an object store returning 200 OK proves durability, not correctness. Server-side validation is the gate that converts an opaque blob into a typed, sized, structurally sound object the rest of your pipeline is allowed to touch.

This page covers the gate itself: how signature detection works underneath, where to place it in the request path, a complete streaming implementation, the policy surface you will need to tune, and the failure modes that only show up once real traffic arrives. It sits inside backend validation and cloud storage architecture, immediately after credentials are issued by S3 presigned URL workflows and immediately before automated virus scanning integration takes over.

Prerequisites #

• [ ] Node 20.11+ (for stable node:test, Blob, and --experimental-strip-types on 22.6+)
• [ ] An object store bucket with two prefixes: pending/ for unverified bytes and public/ for published ones
• [ ] A lifecycle rule that expires pending/ after one day — see setting up S3 lifecycle rules for temporary uploads
• [ ] A row per upload with a status column you can update conditionally (Postgres in the examples)
• [ ] sharp 0.33+ if you accept images, for the structural pass described in validating image dimensions and pixel bombs server-side
• [ ] A reverse proxy configured to let your application see the request body, not buffer it to disk first

How it works #

Validation is not one check. It is a short ladder of increasingly expensive questions, and the useful mental model is that each rung costs the attacker more than the one below it.

Three claims, three levels of proof #

An upload arrives carrying three separate claims, and none of them are evidence. The extension is part of a filename the user typed. The Content-Type header is filled in by the browser from the operating system’s extension-to-type map — the reasons that map lies are worked through in why browser MIME types are unreliable. The bytes are the only thing an attacker cannot trivially rewrite while still getting the file to work for its intended purpose.

The practical consequence is that the declared type is still useful — as an input, never as a verdict. You compare it against what you sniffed, and a disagreement is itself a signal worth logging. A user whose Windows machine reports image/pjpeg for a JPEG is a compatibility quirk. A request that declares image/png and carries <?php is an attack, and the two look identical if you only record “rejected”.

What a signature match actually compares #

A signature is a triple: an offset, a byte sequence, and an optional mask. Matching is a fixed-length memcmp at that offset, not a search — searching would let an attacker prepend arbitrary junk and still match. Three details separate a toy matcher from one that survives contact with real files.

Offsets are not always zero. ISO base media files (MP4, MOV, M4A, HEIC) put a four-byte box length first and the ASCII ftyp at offset 4. If your table only checks offset 0 you will reject every MP4 you receive.

Some bytes must be ignored. A RIFF container starts 52 49 46 46, then four bytes of chunk length that vary per file, then the form type — 57 45 42 50 for WebP. Without a mask you cannot express “these four bytes are anything”.

Longest match wins. 47 49 46 38 matches GIF, and PNG’s eight-byte signature is strictly more specific than any three-byte prefix. Sort candidates by signature length and take the longest, or a short signature that happens to alias will shadow a precise one.

Containers make the format a two-step lookup #

Several formats you care about share a single outer signature, so a match tells you the envelope and not the content. Every OOXML document (.docx, .xlsx, .pptx), every OpenDocument file, every JAR, EPUB and APK starts with the ZIP local file header 50 4B 03 04. Accepting application/zip because you wanted to accept Word documents is how a JAR walks in.

The disambiguation is cheap because ZIP writers put a predictable first entry in the archive. The bytes at offset 26 and 28 give the filename and extra-field lengths, the filename itself starts at offset 30, and that name is enough: OOXML writes [Content_Types].xml first, OpenDocument writes an uncompressed mimetype entry whose contents are the media type string, and JAR/APK write something under META-INF/. All of that lands inside the first 4 KiB, which is why the header window is sized the way it is.

ISO base media files need the same treatment one level down. The ftyp box at offset 4 is followed by a four-character brand — isom, mp42, qt , heic, avif. Accepting the box while ignoring the brand means an HEIC lands in a pipeline that expected MP4 video. If you only support a subset, keep an explicit brand allowlist next to the signature.

Where the gate sits in the request path #

There are exactly two placements, and the choice follows from whether bytes traverse your application at all.

If the browser posts to your server — the classic multipart/form-data submission parsed as described in parsing multipart/form-data in a Node server — the gate is a Transform in the middle of the pipe. You get to reject before the last byte arrives, which is the single biggest advantage of this shape: a 2 GB upload of a disguised executable can be killed after 4 KiB.

If the browser uploads straight to the bucket, as in direct-to-cloud upload patterns, the object exists before you can look at it. The gate becomes an event consumer reading the head of the object with a ranged GET, and the enforcement you lose at the network edge you regain with an S3 POST policy that caps size at signing time — see enforcing upload size limits with S3 POST policies.

Most production systems end up with both: the event-driven gate is the authoritative one because it is the only one an attacker cannot route around, and the proxy gate exists to give honest users a fast, specific error.

Step-by-step implementation #

1. Declare the policy table #

Everything downstream reads from one object. Keeping limits, accepted signatures and canonical extensions in a single structure means a new format is a data change, not a code change, and it gives your tests something to enumerate.

// policy.ts
export interface FamilyPolicy {
  /** Sniffed types this declared family may resolve to. */
  readonly accept: readonly string[];
  /** Canonical extension we write — never the one the user sent. */
  readonly extension: string;
  readonly maxBytes: number;
  /** Only meaningful for raster formats. */
  readonly maxPixels?: number;
  readonly requireStructuralParse: boolean;
}

export interface Policy {
  readonly headBytes: number;
  readonly families: Readonly<Record<string, FamilyPolicy>>;
}

const MB = 1024 * 1024;

export const UPLOAD_POLICY: Policy = {
  headBytes: 4096,
  families: {
    'image/jpeg': { accept: ['image/jpeg'], extension: 'jpg', maxBytes: 25 * MB, maxPixels: 50_000_000, requireStructuralParse: true },
    'image/png': { accept: ['image/png'], extension: 'png', maxBytes: 25 * MB, maxPixels: 50_000_000, requireStructuralParse: true },
    'image/webp': { accept: ['image/webp'], extension: 'webp', maxBytes: 25 * MB, maxPixels: 50_000_000, requireStructuralParse: true },
    'application/pdf': { accept: ['application/pdf'], extension: 'pdf', maxBytes: 40 * MB, requireStructuralParse: false },
    'video/mp4': { accept: ['video/mp4'], extension: 'mp4', maxBytes: 2048 * MB, requireStructuralParse: false },
  },
};

/** Strips `; charset=…` and normalises case before the lookup. */
export function resolveFamily(declared: string, policy: Policy): FamilyPolicy | null {
  const base = declared.split(';')[0].trim().toLowerCase();
  return policy.families[base] ?? null;
}

resolveFamily exists because real clients send image/jpeg; charset=UTF-8 and IMAGE/JPEG. A bare policy.families[req.headers['content-type']] lookup rejects both, and you will spend an afternoon working out why one mobile SDK’s uploads all fail.

2. Build the offset-aware sniffer #

// sniff.ts
export interface Signature {
  readonly mime: string;
  readonly offset: number;
  readonly bytes: readonly number[];
  /** 0xff = compare this byte, 0x00 = ignore it. */
  readonly mask?: readonly number[];
}

const SIGNATURES: readonly Signature[] = [
  { mime: 'image/jpeg', offset: 0, bytes: [0xff, 0xd8, 0xff] },
  { mime: 'image/png', offset: 0, bytes: [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a] },
  { mime: 'image/gif', offset: 0, bytes: [0x47, 0x49, 0x46, 0x38] },
  { mime: 'application/pdf', offset: 0, bytes: [0x25, 0x50, 0x44, 0x46, 0x2d] },
  { mime: 'application/zip', offset: 0, bytes: [0x50, 0x4b, 0x03, 0x04] },
  { mime: 'video/mp4', offset: 4, bytes: [0x66, 0x74, 0x79, 0x70] },
  {
    mime: 'image/webp',
    offset: 0,
    bytes: [0x52, 0x49, 0x46, 0x46, 0x00, 0x00, 0x00, 0x00, 0x57, 0x45, 0x42, 0x50],
    mask: [0xff, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff],
  },
];

const LATIN1 = new TextDecoder('latin1');

function matches(head: Uint8Array, sig: Signature): boolean {
  if (head.length < sig.offset + sig.bytes.length) return false;
  for (let i = 0; i < sig.bytes.length; i += 1) {
    const m = sig.mask ? sig.mask[i] : 0xff;
    if (m === 0) continue;
    if ((head[sig.offset + i] & m) !== (sig.bytes[i] & m)) return false;
  }
  return true;
}

/** ZIP is an envelope: read the first entry name to find out what is inside. */
export function refineZipContainer(head: Uint8Array): string {
  if (head.length < 30) return 'application/zip';
  const view = new DataView(head.buffer, head.byteOffset, head.byteLength);
  const uncompressedSize = view.getUint32(22, true);
  const nameLength = view.getUint16(26, true);
  const extraLength = view.getUint16(28, true);
  const nameEnd = 30 + nameLength;
  if (nameEnd > head.length) return 'application/zip';

  const name = LATIN1.decode(head.subarray(30, nameEnd));
  if (name === '[Content_Types].xml') return 'application/vnd.openxmlformats-officedocument';
  if (name.startsWith('META-INF/')) return 'application/java-archive';
  if (name === 'mimetype') {
    const start = nameEnd + extraLength;
    const end = start + Math.min(uncompressedSize, 128);
    const declared = LATIN1.decode(head.subarray(start, end)).trim();
    if (declared.startsWith('application/vnd.oasis.opendocument')) return declared;
  }
  return 'application/zip';
}

export function sniff(head: Uint8Array): string | null {
  let best: Signature | null = null;
  for (const sig of SIGNATURES) {
    if (!matches(head, sig)) continue;
    if (best === null || sig.bytes.length > best.bytes.length) best = sig;
  }
  if (best === null) return null;
  return best.mime === 'application/zip' ? refineZipContainer(head) : best.mime;
}

Run it against a fixture directory and you should see the container refinement fire:

$ node --input-type=module -e "import {sniff} from './sniff.js'; import {readFileSync} from 'node:fs'; \
  for (const f of ['report.docx','clip.webp','shell.php.jpg']) \
    console.log(f, '→', sniff(readFileSync('fixtures/'+f).subarray(0, 4096)))"
report.docx → application/vnd.openxmlformats-officedocument
clip.webp → image/webp
shell.php.jpg → null

Hand-rolling the table is right when you accept five or six formats. Past roughly twenty, or once you need charset detection and encoding heuristics for text, move to the real signature database — the binding, its build requirements and its async pitfalls are covered in validating file signatures with libmagic in Node.js.

3. Gate the stream and abort early #

The gate buffers up to headBytes, decides once, then becomes a pass-through. It also counts bytes so the size cap is enforced continuously rather than trusting Content-Length.

// gate.ts
import { Transform, type TransformCallback } from 'node:stream';
import { sniff } from './sniff.js';
import { resolveFamily, type FamilyPolicy, type Policy } from './policy.js';

export class ValidationError extends Error {
  constructor(readonly code: string, readonly status: number, message: string) {
    super(message);
    this.name = 'ValidationError';
  }
}

export class SniffGate extends Transform {
  readonly family: FamilyPolicy;
  detected: string | null = null;
  bytesSeen = 0;

  #chunks: Buffer[] = [];
  #buffered = 0;
  #decided = false;

  constructor(declared: string, private readonly policy: Policy) {
    super({ highWaterMark: 64 * 1024 });
    const family = resolveFamily(declared, policy);
    if (family === null) {
      throw new ValidationError('UNSUPPORTED_TYPE', 415, `declared type "${declared}" is not accepted`);
    }
    this.family = family;
  }

  _transform(chunk: Buffer, _encoding: BufferEncoding, cb: TransformCallback): void {
    this.bytesSeen += chunk.length;
    if (this.bytesSeen > this.family.maxBytes) {
      cb(new ValidationError('PAYLOAD_TOO_LARGE', 413, `exceeded ${this.family.maxBytes} bytes`));
      return;
    }
    if (this.#decided) {
      cb(null, chunk);
      return;
    }
    this.#chunks.push(chunk);
    this.#buffered += chunk.length;
    if (this.#buffered < this.policy.headBytes) {
      cb();
      return;
    }
    const head = Buffer.concat(this.#chunks, this.#buffered);
    this.#chunks = [];
    try {
      this.#decide(head);
    } catch (err) {
      cb(err as Error);
      return;
    }
    cb(null, head);
  }

  _flush(cb: TransformCallback): void {
    if (this.#decided) {
      cb();
      return;
    }
    // Short file: we never reached headBytes, so decide on what we have.
    const head = Buffer.concat(this.#chunks, this.#buffered);
    this.#chunks = [];
    try {
      this.#decide(head);
    } catch (err) {
      cb(err as Error);
      return;
    }
    cb(null, head);
  }

  #decide(head: Buffer): void {
    const actual = sniff(head);
    if (actual === null) {
      throw new ValidationError('UNKNOWN_SIGNATURE', 415, `no signature matched the first ${head.length} bytes`);
    }
    if (!this.family.accept.includes(actual)) {
      throw new ValidationError('TYPE_MISMATCH', 415, `sniffed ${actual}, which the declared family does not accept`);
    }
    this.detected = actual;
    this.#decided = true;
  }
}

Wire it into a proxy handler. The hash is computed on the way past, so you never re-read the object to fingerprint it — the same trick the browser can use up front in computing file checksums in the browser with Web Crypto, which lets you compare the two and detect corruption in transit.

// handler.js — proxy upload: validate while streaming to S3
import { pipeline } from 'node:stream/promises';
import { PassThrough } from 'node:stream';
import { createHash, randomUUID } from 'node:crypto';
import { S3Client } from '@aws-sdk/client-s3';
import { Upload } from '@aws-sdk/lib-storage';
import { SniffGate, ValidationError } from './gate.js';
import { UPLOAD_POLICY } from './policy.js';

const s3 = new S3Client({});
const BUCKET = process.env.UPLOAD_BUCKET;

function fail(res, err) {
  if (err instanceof ValidationError) {
    res.status(err.status).json({ error: err.code, message: err.message });
  } else {
    res.status(500).json({ error: 'INTERNAL', message: 'upload could not be stored' });
  }
}

export async function handleUpload(req, res) {
  const declared = req.headers['content-type'] ?? 'application/octet-stream';
  const uploadId = randomUUID();
  const hash = createHash('sha256');

  let gate;
  try {
    gate = new SniffGate(declared, UPLOAD_POLICY);
  } catch (err) {
    fail(res, err);       // 415 before a single body byte is read
    return;
  }

  const body = new PassThrough();
  body.on('data', (chunk) => hash.update(chunk));

  const upload = new Upload({
    client: s3,
    params: { Bucket: BUCKET, Key: `pending/${uploadId}`, Body: body },
    partSize: 8 * 1024 * 1024,
    queueSize: 4,
  });

  try {
    // pipeline() destroys `body` on failure, which rejects upload.done().
    await Promise.all([pipeline(req, gate, body), upload.done()]);
  } catch (err) {
    await upload.abort().catch(() => {});
    fail(res, err);
    return;
  }

  res.status(202).json({
    uploadId,
    contentType: gate.detected,
    sizeBytes: gate.bytesSeen,
    sha256: hash.digest('hex'),
  });
}

A rejected upload produces a log line worth alerting on, because the interesting part is the disagreement rather than the rejection:

warn  upload.rejected code=TYPE_MISMATCH declared=image/png sniffed=application/zip
      bytes_read=4096 upload_id=6f1b6e2a-1d5f-4a0a-9f01-2f3f2a9c77d1 ip=203.0.113.44

4. Parse the structure, not just the header #

A valid signature only proves the first few bytes are well-formed. The structural pass runs on the complete object, which means it belongs in a worker after the bytes have landed, not in the request path. For raster images the parse is header-only inside libvips, so it is far cheaper than a decode.

// structure.ts
import sharp from 'sharp';
import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3';
import { ValidationError } from './gate.js';
import { UPLOAD_POLICY } from './policy.js';

const s3 = new S3Client({});

export interface ImageFacts {
  readonly width: number;
  readonly height: number;
  readonly pages: number;
}

export async function inspectImage(bytes: Buffer, maxPixels: number): Promise<ImageFacts> {
  const meta = await sharp(bytes, { limitInputPixels: maxPixels, failOn: 'error' }).metadata();
  const width = meta.width ?? 0;
  const height = meta.height ?? 0;
  const pages = meta.pages ?? 1;
  if (width === 0 || height === 0) {
    throw new ValidationError('UNDECODABLE', 422, 'image header declares no usable dimensions');
  }
  if (width * height * pages > maxPixels) {
    throw new ValidationError('PIXEL_BUDGET', 422, `${width}x${height}x${pages} exceeds ${maxPixels} pixels`);
  }
  return { width, height, pages };
}

export async function verifyPendingObject(bucket: string, key: string, detected: string) {
  const family = UPLOAD_POLICY.families[detected];
  if (family === undefined) {
    throw new ValidationError('UNSUPPORTED_TYPE', 415, `no policy entry for ${detected}`);
  }
  if (!family.requireStructuralParse) return { detected, facts: null };

  const object = await s3.send(new GetObjectCommand({ Bucket: bucket, Key: key }));
  const bytes = Buffer.from(await object.Body.transformToByteArray());
  const facts = await inspectImage(bytes, family.maxPixels ?? 50_000_000);
  return { detected, facts };
}

Note the pages multiplication: an animated WebP or GIF that is 800 × 600 per frame with 4,000 frames is 1.9 gigapixels even though each frame is trivial. The full treatment of decompression bombs, including why limitInputPixels is a second independent gate rather than a duplicate of the first, is in validating image dimensions and pixel bombs server-side; the archive equivalent lives in detecting and blocking zip bomb uploads. The dimensions you extract here are worth persisting rather than recomputing, which is the argument made in storing image dimensions and duration metadata.

5. Derive the storage key from what you detected #

The filename is untrusted input that ends up in a path, a Content-Disposition header, and probably a log aggregator. Rebuild it rather than sanitising it in place.

// key.ts
import { randomUUID } from 'node:crypto';
import { basename, extname } from 'node:path';

const UNSAFE = /[^a-z0-9._-]+/g;

export function safeKey(prefix: string, originalName: string, extension: string): string {
  const stem = basename(originalName, extname(originalName))
    .normalize('NFC')
    .toLowerCase()
    .replace(UNSAFE, '-')
    .replace(/^-+|-+$/g, '')
    .slice(0, 64);
  const label = stem.length > 0 ? stem : 'file';
  return `${prefix}/${randomUUID()}/${label}.${extension}`;
}

Four things are happening. basename collapses any traversal attempt, so ../../etc/passwd becomes passwd. normalize('NFC') stops two visually identical keys differing in byte representation. The character allowlist removes control characters, % and #, all of which produce keys that are legal in S3 but painful in a URL. And the extension comes from family.extension, never from the upload, so a detected image/png is always written as .png regardless of what the user called it.

> safeKey('public', '../../etc/passwd', 'png')
'public/2c1e4b17-cb6d-4d1f-9a52-0ad55f1b8f4e/passwd.png'
> safeKey('public', 'Ünïcödé  photo (1).JPG', 'jpg')
'public/8b0a1f22-8f4a-4c2a-9d63-7d1e1d2c40aa/-n-c-d-photo-1-.jpg'

6. Flip the object to visible #

The last step is a single guarded state transition. Copy to the public prefix first — the copy is idempotent and server-side, so retrying it costs nothing — then update the row only if it is still in the state you expect.

// publish.ts
import { S3Client, CopyObjectCommand, DeleteObjectCommand } from '@aws-sdk/client-s3';
import { Pool } from 'pg';

const s3 = new S3Client({});
const pool = new Pool({ connectionString: process.env.DATABASE_URL });

export interface PublishInput {
  readonly uploadId: string;
  readonly bucket: string;
  readonly pendingKey: string;
  readonly finalKey: string;
  readonly detected: string;
  readonly sizeBytes: number;
  readonly sha256: string;
}

export async function publish(input: PublishInput): Promise<{ published: boolean }> {
  await s3.send(new CopyObjectCommand({
    Bucket: input.bucket,
    Key: input.finalKey,
    CopySource: `${input.bucket}/${input.pendingKey}`,
    ContentType: input.detected,
    ContentDisposition: 'attachment',
    MetadataDirective: 'REPLACE',
    Metadata: { 'sniffed-type': input.detected, sha256: input.sha256 },
  }));

  const result = await pool.query(
    `UPDATE uploads
        SET status = 'published', storage_key = $2, content_type = $3,
            size_bytes = $4, sha256 = $5, published_at = now()
      WHERE id = $1 AND status = 'validating'
      RETURNING id`,
    [input.uploadId, input.finalKey, input.detected, input.sizeBytes, input.sha256],
  );

  if (result.rowCount === 0) {
    // A redelivered event already advanced this row. The copy above was a no-op.
    return { published: false };
  }

  await s3.send(new DeleteObjectCommand({ Bucket: input.bucket, Key: input.pendingKey }));
  return { published: true };
}

MetadataDirective: 'REPLACE' is load-bearing: without it the copy inherits the Content-Type the client set at upload time, which is precisely the value you just spent five steps refusing to trust. Setting ContentDisposition: 'attachment' on the stored object is a cheap second line of defence for anything you might one day serve from a domain that shares cookies with your application.

Configuration reference #

Error taxonomy and response contract #

Return the code, not the internals. Every message below is safe to show a caller because none of them reveal a bucket name, a key layout or a stack frame — but the structured log line behind each one carries all three.

The 415-versus-422 split matters more than it looks. 415 means “I will never accept this media type”, so a client can stop retrying immediately and say so in the UI. 422 means “the type is right, this instance of it is not”, which is a different message to the user. Emitting 400 for both is the common shortcut and it makes client-side error handling guesswork — the recovery patterns that depend on this distinction are set out in handling 413 and 507 errors during uploads.

Edge cases and gotchas #

Files shorter than the header window #

Your _transform never reaches headBytes for a 900-byte file, so the decision happens in _flush. Forget that branch and short uploads hang until the socket times out. Two related traps: a zero-byte upload produces an empty buffer where sniff correctly returns null, and a three-byte file containing exactly FF D8 FF matches the JPEG signature perfectly while being no kind of image at all. Signature matching cannot catch that; the structural pass in step 4 is what does.

Text formats have no magic number #

SVG, CSV, JSON and plain text have no fixed header — an SVG may begin with an XML declaration, a DOCTYPE, a BOM, or whitespace followed by <svg. If you accept them, sniff returning null is expected and you need a separate branch: parse the document with a real XML parser, reject external entity declarations, and reject <script>, <foreignObject> and event-handler attributes. Then serve it with Content-Disposition: attachment from a domain that shares no cookies with your application, because an SVG rendered inline is script execution in your origin.

Polyglots: one file, two valid headers #

A ZIP archive’s index lives at the end of the file, so appending a complete ZIP to a valid PNG produces bytes that image libraries read as a PNG and archive tools read as an archive. Head sniffing cannot detect this by construction. The durable mitigation is never serving the original bytes for formats you transform anyway: re-encode every image through a derivative pipeline, as in building an image derivative pipeline with Sharp, and serve only the derivatives. Re-encoding drops the appended payload because the encoder writes a fresh container.

The extension is not a function of the MIME type #

image/jpeg maps to .jpg and .jpeg; video/mp4 legitimately covers audio-only .m4a; application/vnd.openxmlformats-officedocument covers .docx, .xlsx and .pptx and the outer signature cannot tell you which. Pick one canonical extension per family that you own and store it, and if you need the real OOXML subtype, read the [Content_Types].xml entry instead of guessing from the archive header.

Declared type arrives from two places at once #

In a multipart/form-data submission the request has a Content-Type of multipart/form-data; boundary=... and each part has its own. The one you want is the part header, and it is exactly as untrusted as the request header. Worse, some HTTP clients append charset=UTF-8 to binary types and a few older Android SDKs send application/octet-stream for everything, which is why resolveFamily strips parameters and why an application/octet-stream family entry that accepts several sniffed types is a pragmatic escape hatch.

Redelivered events and double validation #

S3 event notifications are at-least-once, and a Lambda that times out after doing its work will be retried. Every side effect in the chain must therefore be idempotent: CopyObject to a deterministic key is naturally so, the guarded UPDATE in step 6 turns the second run into a no-op, and object tagging gives you a cheap “already verified” check that costs one GetObjectTagging instead of a full re-parse. The quarantine side needs the same discipline — see quarantine bucket patterns for infected uploads.

Rejecting early still leaves the client sending #

When you destroy the request stream after 4 KiB, an HTTP/1.1 client with 2 GB still to send does not necessarily stop. Node resets the connection, which some clients surface as a network error rather than your carefully crafted 415, and the response body can be lost entirely. Two mitigations: keep your maxBytes below the proxy’s own limit (see raising Nginx and Cloudflare upload size limits) so the failure is yours to shape, and prefer Expect: 100-continue or a pre-flight metadata call for large uploads so the rejection happens before the body starts. Streaming request bodies, covered in streaming file uploads in Node.js with Web Streams, make the abort semantics considerably cleaner.

Verification #

Start with a fixture table. Six files, one assertion each, and the table doubles as documentation of what you accept:

// sniff.test.ts — node --test --experimental-strip-types sniff.test.ts
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { sniff } from './sniff.js';

const CASES: ReadonlyArray<readonly [string, string | null]> = [
  ['fixtures/photo.jpg', 'image/jpeg'],
  ['fixtures/logo.png', 'image/png'],
  ['fixtures/clip.webp', 'image/webp'],
  ['fixtures/report.docx', 'application/vnd.openxmlformats-officedocument'],
  ['fixtures/app.jar', 'application/java-archive'],
  ['fixtures/shell.php.jpg', null],
];

for (const [path, expected] of CASES) {
  test(`sniff ${path}`, () => {
    const head = readFileSync(path).subarray(0, 4096);
    assert.equal(sniff(head), expected);
  });
}

test('a three-byte JPEG prefix still matches, which is why parsing matters', () => {
  assert.equal(sniff(Uint8Array.from([0xff, 0xd8, 0xff])), 'image/jpeg');
});

Then prove the HTTP contract end to end. Build a hostile fixture and check the exact status and body:

printf '<?php eval($_GET["c"]); ?>' > fixtures/shell.php.jpg

curl -i -X POST https://api.example.com/uploads \
  -H 'Content-Type: image/jpeg' \
  --data-binary @fixtures/shell.php.jpg

HTTP/1.1 415 Unsupported Media Type
content-type: application/json; charset=utf-8

{"error":"UNKNOWN_SIGNATURE","message":"no signature matched the first 26 bytes"}

Prove the early abort actually saves bandwidth by sending a large disguised file and reading back how much of it left the machine. On a healthy gate size_upload lands in the tens of kilobytes rather than the full file size:

head -c 200000000 /dev/urandom > /tmp/big.bin
curl -s -o /dev/null -X POST https://api.example.com/uploads \
  -H 'Content-Type: image/png' --data-binary @/tmp/big.bin \
  -w 'status=%{http_code} uploaded=%{size_upload} time=%{time_total}\n'

Finally, confirm the published object carries the type you detected rather than the one the client declared:

aws s3api head-object --bucket "$UPLOAD_BUCKET" \
  --key "public/2c1e4b17-cb6d-4d1f-9a52-0ad55f1b8f4e/passwd.png" \
  --query '{type:ContentType,meta:Metadata,len:ContentLength}'

If ContentType comes back as whatever the browser sent, your copy is missing MetadataDirective: 'REPLACE'. Round it off with a SQL check that no row is stuck: SELECT status, count(*) FROM uploads WHERE created_at > now() - interval '1 hour' GROUP BY status should show no validating rows older than your worker timeout.

Frequently Asked Questions