Quarantine Bucket Patterns for Infected Uploads #

Give an upload three possible homes — an incoming bucket nobody can read, a clean bucket, and a quarantine bucket with retention turned on — and make the move between them depend on an object tag that only the scanner role is allowed to write.

This article sits under automated virus scanning integration within backend validation and cloud storage architecture. It assumes the scanner itself already exists — built either as a container running clamd or as serverless virus scanning with AWS Lambda — and concerns itself only with where the bytes live and who is permitted to touch them.

When to use this approach #

• You accept uploads from untrusted users and serve them back to other users, so a single missed executable is a stored-XSS or malware-distribution incident rather than a bad row in a table.
• Uploads land directly in object storage via presigned URLs, so there is no request-path server that can hold the file until a verdict exists.
• You need to keep infected samples for weeks — for incident review, for a customer support conversation, or because a compliance framework says so — instead of deleting them on detection.

If files pass through your own API before storage, a single bucket with a pending/ prefix is usually enough. The three-bucket split earns its complexity when the write path and the read path are different principals in different accounts.

Prerequisites #

1. AWS SDK v3 (@aws-sdk/client-s3 3.600.0 or later) on Node 20+, and aws-cli 2.x for the verification steps.
2. Three buckets with S3 Block Public Access enabled at the account level: acme-uploads-incoming, acme-uploads-clean, acme-uploads-quarantine.
3. Versioning enabled on the quarantine bucket, plus Object Lock (which requires versioning) if you want retention that survives a compromised admin.
4. Four IAM identities: the browser’s presigned-URL grant, an app-api role that serves files, an av-scanner role, and a break-glass forensics role.
5. A separate KMS key per bucket. One key for all three defeats the point.

The three-bucket state machine #

An object has exactly one live location at any moment, and the transitions are one-way. It lands in incoming carrying scan-status=pending. The scanner reads it, decides, and performs a copy-then-delete into either clean or quarantine. There is no path back from quarantine to clean that does not involve a human deliberately re-running the promotion.

The property worth defending is narrow: no principal that can serve bytes to an end user can also read the incoming bucket. Everything else in the design exists to keep that true under partial failure. If the scanner crashes mid-run, the object stays in incoming, unreadable. If the promotion copy succeeds but the delete fails, you have a duplicate in clean and a stale source, which a lifecycle rule sweeps up — the same expiry mechanics described in setting up S3 lifecycle rules for temporary uploads.

The IAM boundary that makes it real #

Prefixes inside one bucket are a naming convention; separate buckets with separate policies and separate KMS keys are an enforcement boundary. Write the permission matrix down before you write any code, because every cell you leave vague becomes an incident later.

Note what the matrix implies about serving. The app-api role reads from clean only, so a bug in your download handler that forgets to check the database cannot reach an unscanned object — the credentials simply do not stretch that far. That is a stronger guarantee than any code review, and it composes well with the short-lived download URLs you get from generating secure presigned URLs with AWS SDK v3.

Implementation #

The tag-conditioned bucket policy #

This is the policy on acme-uploads-incoming. It denies reads unless the object carries scan-status=clean, denies tag writes from anyone but the scanner, and forces every upload to arrive tagged pending.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyReadUnlessScanStatusClean",
      "Effect": "Deny",
      "Principal": "*",
      "Action": ["s3:GetObject", "s3:GetObjectVersion"],
      "Resource": "arn:aws:s3:::acme-uploads-incoming/*",
      "Condition": {
        "StringNotEquals": { "s3:ExistingObjectTag/scan-status": "clean" },
        "ArnNotLike": {
          "aws:PrincipalArn": [
            "arn:aws:iam::111122223333:role/av-scanner",
            "arn:aws:sts::111122223333:assumed-role/av-scanner/*"
          ]
        }
      }
    },
    {
      "Sid": "OnlyScannerWritesTheVerdict",
      "Effect": "Deny",
      "Principal": "*",
      "Action": ["s3:PutObjectTagging", "s3:DeleteObjectTagging"],
      "Resource": "arn:aws:s3:::acme-uploads-incoming/*",
      "Condition": {
        "ArnNotLike": {
          "aws:PrincipalArn": [
            "arn:aws:iam::111122223333:role/av-scanner",
            "arn:aws:sts::111122223333:assumed-role/av-scanner/*"
          ]
        }
      }
    },
    {
      "Sid": "UploadsMustArriveTaggedPending",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::acme-uploads-incoming/*",
      "Condition": {
        "StringNotEquals": { "s3:RequestObjectTag/scan-status": "pending" }
      }
    }
  ]
}

The first statement carries the whole design. s3:ExistingObjectTag/scan-status resolves the tag already on the object at request time, and because StringNotEquals is a negated operator it evaluates to true when the key is absent entirely. An object with no tags at all is therefore denied, which is the direction you want a policy to fail in. Had you written this as an Allow with StringEquals, an untagged object would fall through to whatever identity policy the caller holds.

The third statement uses s3:RequestObjectTag/scan-status, which inspects the x-amz-tagging header on the incoming PutObject. Without it an uploader with a presigned URL could set x-amz-tagging: scan-status=clean at creation time and skip the scanner completely, because tags applied during PutObject are not gated by the s3:PutObjectTagging action. The two statements only close the hole together.

Apply statements one and two verbatim to acme-uploads-clean with its own ARN. On acme-uploads-quarantine, add the forensics role to the ArnNotLike list in the read deny — otherwise nobody can ever look at the sample you carefully retained.

The promotion function #

import {
  S3Client,
  CopyObjectCommand,
  DeleteObjectCommand,
  HeadObjectCommand,
} from "@aws-sdk/client-s3";
import { SNSClient, PublishCommand } from "@aws-sdk/client-sns";

const s3 = new S3Client({});
const sns = new SNSClient({});

const INCOMING = process.env.INCOMING_BUCKET as string;
const CLEAN = process.env.CLEAN_BUCKET as string;
const QUARANTINE = process.env.QUARANTINE_BUCKET as string;
const ALERT_TOPIC = process.env.ALERT_TOPIC_ARN as string;

export type Verdict =
  | { status: "clean" }
  | { status: "infected"; signature: string };

export async function settle(key: string, verdict: Verdict): Promise<string> {
  // 1. Pin the exact bytes we scanned. If someone overwrote the object
  //    while the scan ran, the copy below fails instead of promoting it.
  const head = await s3.send(
    new HeadObjectCommand({ Bucket: INCOMING, Key: key }),
  );

  const clean = verdict.status === "clean";
  const destination = clean ? CLEAN : QUARANTINE;
  const scannedAt = new Date().toISOString();

  // 2. Tag values allow letters, digits, space and + - = . _ : / @ only.
  const tagging = clean
    ? `scan-status=clean&scanned-at=${scannedAt}`
    : `scan-status=infected&scanned-at=${scannedAt}` +
      `&signature=${encodeURIComponent(verdict.signature)}`;

  // 3. Copy and tag in ONE request. The object never exists untagged
  //    in a bucket that anything is allowed to read.
  await s3.send(
    new CopyObjectCommand({
      Bucket: destination,
      Key: key,
      CopySource: `${INCOMING}/${encodeURIComponent(key)}`,
      CopySourceIfMatch: head.ETag,
      MetadataDirective: "COPY",
      TaggingDirective: "REPLACE",
      Tagging: tagging,
      ServerSideEncryption: "aws:kms",
      SSEKMSKeyId: clean
        ? (process.env.CLEAN_KMS_KEY as string)
        : (process.env.QUARANTINE_KMS_KEY as string),
    }),
  );

  // 4. Only now does the source disappear.
  await s3.send(new DeleteObjectCommand({ Bucket: INCOMING, Key: key }));

  if (!clean) {
    await sns.send(
      new PublishCommand({
        TopicArn: ALERT_TOPIC,
        Subject: `Infected upload quarantined: ${key.slice(0, 60)}`,
        Message: JSON.stringify(
          { key, signature: verdict.signature, scannedAt, bucket: QUARANTINE },
          null,
          2,
        ),
      }),
    );
  }

  return `s3://${destination}/${key}`;
}

Line-by-line on the parameters that matter #

• HeadObjectCommand first gives you the current ETag. Without it the promotion is a time-of-check-to-time-of-use bug: the scanner reads version A, an attacker overwrites with version B, and you promote B unscanned.
• CopySourceIfMatch: head.ETag turns that into a hard failure. A mismatch returns HTTP 412 with PreconditionFailed: At least one of the pre-conditions you specified did not hold, and the object simply stays in incoming for the next scan.
• TaggingDirective: "REPLACE" plus Tagging is the atomic part. COPY would carry the source’s scan-status=pending across, and tagging afterwards would open the window the next section is about.
• MetadataDirective: "COPY" preserves the original Content-Type and any custom x-amz-meta-* headers you set at upload time — losing the content type here is how promoted images start downloading as application/octet-stream.
• SSEKMSKeyId re-encrypts under the destination bucket’s key. The quarantine key’s policy should not grant kms:Decrypt to app-api at all, so even a bucket policy mistake leaves the ciphertext unreadable.
• encodeURIComponent(key) on CopySource encodes slashes as %2F, which S3 accepts and which is the only reliable way to handle keys containing +, # or ?.
• The delete is last, and unconditional. If the copy threw, this line never runs and the object is retried.

Closing the read-before-tag race #

The failure that actually bites people is ordering. Copy the object into the serving bucket, then tag it in a second call, and for the duration of that second call the object exists in a bucket your application can read while carrying no verdict at all. On a warm Lambda in the same region that window is 30–80 ms; under retry it can be seconds. It is small, it is real, and it is trivially removable.

The same reasoning applies at the front of the pipeline. The presigned URL you hand the browser must sign the x-amz-tagging: scan-status=pending header, so the object is born tagged. If you instead tag it from a Lambda triggered by ObjectCreated, there is a window in which the object is untagged — and although the negated-operator behaviour above means it is still denied, you have made your safety depend on a subtlety rather than on the object’s own state.

One consequence you must design around: S3 object tags are eventually consistent, even though the object body itself is read-after-write consistent. Immediately after promotion, a GetObject can still be evaluated against the pre-copy tag state and be denied. In practice this resolves within a second or two. Do not paper over it with a retry loop in the download handler; instead, let the promotion function write the row that marks the file available and have the client poll or subscribe to that — the pattern in streaming upload progress with Server-Sent Events works unchanged for a “scanning…” state.

Retention and forensics on the quarantine bucket #

Deleting an infected file feels tidy and destroys the only evidence you have. Keep it, and make keeping it non-optional:

• Versioning plus Object Lock in governance mode, with a default retention of 90 days. Governance mode means a role holding s3:BypassGovernanceRetention can still remove an object; compliance mode means nobody can, including the account root, until the clock runs out. Start with governance unless a regulator has told you otherwise, because compliance mode has no undo for your own mistakes.
• A lifecycle rule expiring objects at 365 days. Lifecycle respects Object Lock, so the rule quietly does nothing until retention lapses and then cleans up — you get both a floor and a ceiling on how long you hold malware.
• Blocked inline access. No CloudFront distribution, no origin access control, no bucket-level s3:GetObject allow for anything except the forensics role.
• Record the verdict alongside the file record, not only as a tag: signature name, engine version, scan duration, and the destination URI. Tags are limited to 10 per object and 256 characters per value, so the durable copy belongs in your metadata store — see how to index file metadata in PostgreSQL for the schema side.

Alerting is the other half. The SNS publish in the promotion function is your primary signal, but it lives in the same code path that might be the thing that broke. Add an EventBridge rule on the quarantine bucket for Object Created as an independent backstop, and a scheduled check that counts objects in incoming older than one hour — a rising count there means the scanner has stalled and uploads are silently failing to appear for users.

Configuration gotchas #

MalformedPolicy: Action does not apply to any resource(s) in statement #

You wrote "Resource": "arn:aws:s3:::acme-uploads-incoming" without the trailing /*. Object-level actions such as s3:GetObject and s3:PutObjectTagging only apply to object ARNs; bucket-level actions such as s3:ListBucket only apply to the bucket ARN. If one statement needs both, list both ARNs — do not merge the actions into a single statement with a single resource.

The deny matches your scanner too #

aws:PrincipalArn conditions are where these policies usually break. If the deny fires against the scanner itself you will see AccessDenied ... with an explicit deny in a resource-based policy in the function logs, and the object never leaves incoming. List both the role ARN and the sts assumed-role wildcard, as the policy above does; with a negated operator such as ArnNotLike, the condition is false — meaning the deny does not apply — only when the principal matches at least one entry. Verify with the IAM policy simulator before you attach it, and keep a second terminal authenticated as an admin while you do.

AccessDenied: There were headers present in the request which were not signed #

Your presigned PUT includes x-amz-tagging in the signature but the browser did not send it, or sent a different value. Whatever you sign, the client must reproduce byte for byte. fetch will not add the header for you:

await fetch(presignedUrl, {
  method: "PUT",
  headers: { "x-amz-tagging": "scan-status=pending" },
  body: file,
});

Remember that x-amz-tagging also needs to be in your bucket’s CORS AllowedHeaders, or the preflight fails before the PUT is ever attempted.

InvalidRequest: The specified copy source is larger than the maximum allowable size for a copy source: 5368709120 #

CopyObject tops out at 5 GB. Above that the promotion must use multipart copy — CreateMultipartUpload on the destination, a series of UploadPartCopy calls with CopySourceRange headers, then CompleteMultipartUpload. Pass the Tagging parameter on CreateMultipartUpload, because there is nowhere to attach it on completion, and the atomicity argument above still holds. Media pipelines hit this constantly; the sizing trade-offs are covered in best practices for handling 500MB file uploads.

Verification #

Prove the boundary with credentials rather than by reading the policy back. This runs against the clean bucket using three named CLI profiles:

#!/usr/bin/env bash
set -u
BUCKET=acme-uploads-clean
KEY="probe/$(date +%s).bin"
head -c 1024 /dev/urandom > probe.bin

aws s3api put-object --bucket "$BUCKET" --key "$KEY" --body probe.bin \
  --tagging 'scan-status=pending' --profile av-scanner

aws s3api get-object --bucket "$BUCKET" --key "$KEY" /dev/null \
  --profile app-api 2>&1 | grep -q 'explicit deny' \
  && echo "PASS: pending object is unreadable"

aws s3api put-object-tagging --bucket "$BUCKET" --key "$KEY" \
  --tagging 'TagSet=[{Key=scan-status,Value=clean}]' --profile app-api 2>&1 \
  | grep -q 'AccessDenied' && echo "PASS: app-api cannot forge a verdict"

aws s3api put-object-tagging --bucket "$BUCKET" --key "$KEY" \
  --tagging 'TagSet=[{Key=scan-status,Value=clean}]' --profile av-scanner
sleep 3
aws s3api get-object --bucket "$BUCKET" --key "$KEY" /dev/null \
  --profile app-api > /dev/null && echo "PASS: clean object is readable"

aws s3api delete-object --bucket "$BUCKET" --key "$KEY" --profile av-scanner
rm -f probe.bin

Four PASS lines means the tag is genuinely load-bearing. Then run the EICAR test string through the real upload path and confirm it lands in quarantine with a signature tag and that the SNS alert arrives. Add both to CI against a throwaway account: bucket policies drift, and the failure is silent until it is not.

Frequently Asked Questions