Parsing multipart/form-data in a Node Server #

Pipe the raw IncomingMessage into busboy, write each file part directly to its destination as it arrives, and set limits so the parser stops feeding you bytes before an oversized upload ever reaches disk.

This article sits under multipart form data explained inside upload fundamentals and browser APIs. The client half — how the browser builds the body you are about to take apart — is covered in implementing multipart/form-data in vanilla JavaScript.

When to use this approach #

• Your origin server terminates the upload itself rather than handing the browser a presigned URL. If you have the choice, weigh it against presigned URL vs server proxy trade-offs first — the cheapest parser is the one you never run.
• Files are large enough that buffering them is a memory risk: anything over a few megabytes, or any endpoint that can be called concurrently by more than a handful of clients.
• You need metadata fields and binary parts in the same request, and you want to reject a bad request while the body is still arriving instead of after it has all landed.

Prerequisites #

1. Node 20.11 or newer (node:stream/promises, Readable.toWeb, and non-deprecated close semantics on requests).
2. npm i busboy@1.6.0. For the S3 variant, @aws-sdk/client-s3 and @aws-sdk/lib-storage v3.
3. A reverse proxy that is not already truncating you — Nginx returns 413 Request Entity Too Large on its own before Node sees a byte unless client_max_body_size is raised, as covered in handling large file size limits.

How a streaming parser sees the body #

busboy is a state machine over a byte stream. It scans for the boundary delimiter taken from the Content-Type header, reads the part headers that follow it, and then emits everything up to the next boundary as a Readable. It never holds a whole part in memory; the only buffer it keeps is roughly the length of the boundary plus the part headers, so a 4 GB upload and a 4 KB upload cost the parser the same.

That design has one consequence people trip over constantly: the part streams are the parser’s back-pressure valve. If you attach a file listener and do not consume the stream, busboy pauses and the request never completes.

Implementation #

One promise-wrapped parser, a node:http server around it, and no buffering anywhere. Save it as server.mjs and run it with node server.mjs.

// server.mjs — Node 20.11+, busboy 1.6.0
import { createServer } from 'node:http';
import { createWriteStream } from 'node:fs';
import { unlink } from 'node:fs/promises';
import { pipeline } from 'node:stream/promises';
import { tmpdir } from 'node:os';
import { join, basename } from 'node:path';
import { randomUUID } from 'node:crypto';
import busboy from 'busboy';

const MAX_FILE_BYTES = 100 * 1024 * 1024; // 100 MB per file
const ALLOWED = new Set(['image/jpeg', 'image/png', 'video/mp4']);

function parseUpload(req) {
  return new Promise((resolve, reject) => {
    let bb;
    try {
      bb = busboy({
        headers: req.headers,        // the boundary is read from here, never guessed
        defParamCharset: 'utf8',     // default is latin1 — mangles non-ASCII filenames
        fileHwm: 256 * 1024,         // 256 KB per part-stream read
        limits: {
          files: 1,
          fileSize: MAX_FILE_BYTES,
          fields: 10,
          fieldSize: 32 * 1024,
          parts: 12,
        },
      });
    } catch (err) {
      err.statusCode = 415;          // thrown synchronously on a bad Content-Type
      reject(err);
      return;
    }

    const fields = Object.create(null);
    const stored = [];
    const pending = [];
    let settled = false;

    const fail = (err, statusCode) => {
      if (settled) return;
      settled = true;
      req.unpipe(bb);
      err.statusCode = statusCode;
      reject(err);
    };

    bb.on('field', (name, value) => { fields[name] = value; });

    bb.on('file', (name, stream, info) => {
      const { filename, mimeType } = info;
      if (!ALLOWED.has(mimeType)) {
        stream.resume();             // drain, or busboy never emits 'close'
        fail(new Error(`Unsupported media type: ${mimeType}`), 415);
        return;
      }

      const target = join(tmpdir(), `${randomUUID()}-${basename(filename)}`);
      let truncated = false;
      stream.on('limit', () => { truncated = true; });

      pending.push(
        pipeline(stream, createWriteStream(target)).then(async () => {
          if (truncated) {
            await unlink(target).catch(() => {});
            throw new Error(`File exceeds ${MAX_FILE_BYTES} bytes: ${filename}`);
          }
          stored.push({ field: name, filename, mimeType, path: target });
        }),
      );
    });

    bb.on('filesLimit', () => fail(new Error('Too many file parts'), 400));
    bb.on('fieldsLimit', () => fail(new Error('Too many fields'), 400));
    bb.on('partsLimit', () => fail(new Error('Too many parts'), 400));
    bb.on('error', (err) => fail(err, 400));

    bb.on('close', () => {
      Promise.all(pending)
        .then(() => { if (!settled) { settled = true; resolve({ fields, files: stored }); } })
        .catch((err) => fail(err, 413));
    });

    req.on('close', () => {
      if (!req.readableEnded) fail(new Error('Client aborted the upload'), 499);
    });

    req.pipe(bb);
  });
}

const server = createServer(async (req, res) => {
  if (req.method !== 'POST' || req.url !== '/upload') {
    res.writeHead(404).end();
    return;
  }
  try {
    const result = await parseUpload(req);
    res.writeHead(201, { 'content-type': 'application/json' });
    res.end(JSON.stringify(result));
  } catch (err) {
    req.resume();                    // drain the rest, or the client sees ECONNRESET
    res.writeHead(err.statusCode ?? 400, { 'content-type': 'application/json' });
    res.end(JSON.stringify({ error: err.message }));
  }
});

server.listen(3000, () => console.log('listening on :3000'));

Line-by-line on the parameters that matter #

• headers: req.headers is the whole boundary story. busboy parses multipart/form-data; boundary=----WebKitFormBoundaryAbc123 out of the header you hand it. Do not reconstruct the delimiter from the body’s first line: a part’s own content can legally contain a line that looks like a boundary, and the browser is free to pick any token up to 70 characters.
• defParamCharset: 'utf8' matters the moment a user uploads résumé.pdf. busboy 1.6 defaults Content-Disposition parameter decoding to latin1, so without this you store résumé.pdf. The RFC 5987 filename*=UTF-8'' form is decoded correctly either way; the plain filename= form is not.
• fileHwm: 256 * 1024 raises the part stream’s high-water mark from the 16 KB stream default. On a 100 MB video that is roughly 400 reads instead of 6,400, which measurably cuts syscall overhead without changing the memory ceiling in any way that matters.
• limits.fileSize is enforced by the parser, not by your handler. Once the counter passes it, busboy stops pushing bytes into that part stream, emits limit on it, and carries on parsing the rest of the body. Nothing throws.
• stream.resume() in the rejection path is mandatory. busboy’s contract is that every emitted file stream must be consumed; discard the contents with resume() if you do not want them, or close never fires and the promise never settles.
• pipeline(stream, createWriteStream(target)) wires up back-pressure and destroys both ends on failure. Using stream.pipe(ws) instead leaks the write stream’s file descriptor when the request aborts mid-part.
• pending plus Promise.all in the close handler is what stops you replying 201 before fsync has happened. busboy’s close means “the body is fully parsed”, not “your writes have flushed”.
• req.resume() in the error path drains whatever the client is still sending. Responding and closing while a 500 MB body is in flight gives the browser ECONNRESET instead of your JSON error.

Enforcing limits before the bytes land #

The distinction that costs people a weekend: a limit hit is not an error. busboy signals it and keeps going, which is deliberate — the parser cannot know whether you want to reject the request or accept it with one part dropped. If you never look at the limit event or the stream’s truncated property, you write a partial file, index it, and return 201.

The same rule applies to files, fields and parts. Exceeding files: 1 fires filesLimit once and every further file part is skipped without an event of its own, so a client uploading three files against a one-file limit gets a perfectly cheerful response describing one file unless you wire filesLimit to a rejection. Set parts as well as files: parts bounds the total number of delimiters the parser will process and is your defence against a body made of a hundred thousand empty parts.

Size limits are a resource guard, not a validation step. Once the bytes are down you still need to confirm they are what they claim to be — validating file signatures with libmagic in Node.js covers the magic-byte check that the Content-Type in a part header can never substitute for.

Streaming a part straight to S3 #

Writing to tmpdir() is fine when a worker will pick the file up locally. If the object’s home is S3, skip the disk entirely and hand the part stream to @aws-sdk/lib-storage, which performs a multipart upload behind the scenes and accepts a stream of unknown length.

import { S3Client } from '@aws-sdk/client-s3';
import { Upload } from '@aws-sdk/lib-storage';

const s3 = new S3Client({ region: process.env.AWS_REGION });

// inside bb.on('file', (name, stream, info) => { ... })
function uploadPart(stream, info) {
  const upload = new Upload({
    client: s3,
    params: {
      Bucket: process.env.UPLOAD_BUCKET,
      Key: `incoming/${randomUUID()}`,
      Body: stream,                  // the part stream itself, not a Buffer
      ContentType: info.mimeType,
    },
    partSize: 5 * 1024 * 1024,       // 5 MB is the S3 minimum for non-final parts
    queueSize: 2,                    // at most 10 MB buffered per request
    leavePartsOnError: false,        // abort the MPU so orphan parts stop billing
  });
  return upload.done();
}

Be honest about the memory: partSize * queueSize is buffered per in-flight upload, so the defaults (5 MB, 4) cost 20 MB per concurrent request. Dropping queueSize to 2 halves that at the price of some parallelism. Sixty concurrent uploads at 10 MB each is 600 MB of resident memory — still bounded, but you have to budget for it. If your traffic profile makes that arithmetic uncomfortable, the answer is usually to stop proxying at all and use direct-to-cloud upload patterns.

What buffering actually costs #

The alternative most codebases start with is a buffering parser: multer with memoryStorage, an express.raw() body, or the Web-standard request.formData(). All of them materialise the entire part before your handler runs, so resident memory scales with file size times concurrency rather than staying flat.

The failure mode is abrupt. On a 1 GB container the process is OOM-killed with exit code 137 and no stack trace; with a larger heap you instead get FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory. Neither is diagnosable from the request logs, because the request that pushed you over is rarely the one that was misbehaving.

The Web-standard route is worth knowing because it is genuinely convenient in Workers, Deno and Bun, and it works in Node too:

import { Readable } from 'node:stream';

async function parseWithWebStandard(req) {
  const res = new Response(Readable.toWeb(req), {
    headers: { 'content-type': req.headers['content-type'] ?? '' },
  });
  const form = await res.formData();
  const file = form.get('file');       // a File object, already fully in memory
  return { name: form.get('album'), size: file.size, type: file.type };
}

Four lines instead of eighty, and the right call for a 200 KB avatar. But formData() has no limits argument at all: undici materialises every part as an in-memory File before the promise resolves, so the only ceiling you can impose is a Content-Length check before you start — and Content-Length is absent on a chunked request. Use it for small, trusted, size-capped endpoints; use busboy for anything a user can point a large file at.

Configuration gotchas #

Error: Multipart: Boundary not found — busboy throws this synchronously from the constructor when Content-Type is multipart/form-data with no boundary= parameter. It is almost always a client that set the header by hand and stripped the browser-generated boundary. Catch it around the busboy() call, as the implementation does, and answer 415; an uncaught throw inside your request handler takes the process down.

Error: Unexpected end of form — emitted on the busboy instance (not thrown) when the socket closes before the closing --boundary-- delimiter arrives. Cause is a client abort, a proxy timeout, or a Content-Length that overstates the body. Treat it as 400, and delete any partial file you have already opened; without an error listener this becomes an unhandled 'error' event and kills the process.

Error: Unsupported content type: application/json — also thrown by the constructor. Guard the route with a req.headers['content-type']?.startsWith('multipart/form-data') check if you would rather return a clearer message than busboy’s.

The request that hangs and returns nothing. No error string, because nothing failed: you listened for file, ignored the stream, and busboy stopped reading. The request sits open until Node’s server.requestTimeout (300,000 ms by default since Node 18) closes it. If a route works for small files in tests and hangs in production, look for a code path that returns early from the file handler without calling resume() or piping. The same symptom shows up client-side as a stalled progress bar, which is why streaming upload progress with server-sent events is worth wiring in before you debug it blind.

Verification #

Start the server, then exercise the happy path, the size limit and the missing boundary:

# 1. Happy path — expect HTTP/1.1 201 and a JSON body listing one file.
curl -i -X POST http://localhost:3000/upload \
  -F "album=holiday-2026" \
  -F "file=@./clip.mp4;type=video/mp4"

# 2. Oversize — 120 MB against a 100 MB limit. Expect 413.
head -c 120000000 /dev/urandom > /tmp/big.mp4
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://localhost:3000/upload \
  -F "file=@/tmp/big.mp4;type=video/mp4"

# 3. Boundary stripped by hand. Expect 415 and the busboy message.
curl -s -X POST http://localhost:3000/upload \
  -H 'Content-Type: multipart/form-data' --data-binary 'not-a-part'
# {"error":"Multipart: Boundary not found"}

Then prove the memory claim rather than trusting it. Log RSS on an interval while eight concurrent 100 MB uploads run; the streaming server should stay flat within a few megabytes of idle:

const idle = process.memoryUsage().rss;
setInterval(() => {
  const mb = (process.memoryUsage().rss / 1024 / 1024).toFixed(1);
  console.log(`rss=${mb}MB delta=${((process.memoryUsage().rss - idle) / 1024 / 1024).toFixed(1)}MB`);
}, 1000).unref();

Confirm afterwards that no orphan files survived a rejected upload: ls /tmp should contain no UUID-prefixed leftovers from test 2.

Frequently Asked Questions