Caching and CDN Headers for Docs Sites

This guide is part of Docs CI/CD & Preview Deployments within Developer Portal Frameworks & UI Setup. It covers the Cache-Control policy a static documentation site needs, why one site-wide rule is always wrong, and how to confirm that a deploy is actually visible to readers. The scenario that brings people here is universal: the pipeline reports a successful deploy, the new page is in the bucket, and readers keep seeing the old one.

Cache policy by asset class Fingerprinted assets cache for a year, HTML revalidates on every request, and the specification file uses a short lifetime with an ETag. the URL tells you the policy URL changes with content main.8f3a1c.js max-age=31536000 immutable safe forever — a new build renames it URL stable, content changes /guides/auth/index.html max-age=0 must-revalidate cheap 304 when unchanged machine-read contract /openapi/api.yaml max-age=300 + ETag stale here breaks SDKs and mocks

Problem & Context

A documentation deploy replaces files at URLs that do not change. /guides/auth/index.html is the same URL before and after the release; only its bytes differ. Every caching layer between the origin and the reader — the CDN, a corporate proxy, the browser — decides whether to re-fetch based on the headers the origin sent the last time it served that URL. If that response said max-age=86400, nothing will ask again for a day, and no amount of successful deploying changes that.

The opposite mistake is equally common and quieter. Setting no-cache on everything makes deploys instantly visible and makes the site slow: every navigation re-downloads the JavaScript bundle, the stylesheet, and the fonts, on every page, for every reader. On a documentation site people navigate a lot, so this shows up directly in the performance budget.

The resolution is not a compromise value in the middle. It is recognising that the build output contains two fundamentally different kinds of URL — ones whose name changes when their content changes, and ones whose name does not — and that each kind has an obviously correct policy.

Step-by-Step Solution

1. Classify every asset

Inspect what your generator actually emitted. Fingerprinted files carry a content hash in the name; everything else does not:

find dist -type f | sed -E 's/.*\.([a-f0-9]{8,})\.(js|css|woff2|png)$/HASHED  &/' \
  | grep -c '^HASHED' 
find dist -name '*.html' | wc -l

34        # hashed assets — cache forever
57        # HTML documents — must revalidate

If that first number is zero, your generator is not fingerprinting and you cannot use the immutable policy at all. Turn on content hashing before continuing; without it, every asset has to revalidate and the site pays for it on every navigation.

2. Cache hashed assets immutably

A fingerprinted URL is a promise that the bytes never change. Serve it with the longest lifetime the specification allows, plus immutable so browsers skip revalidation even on a reload:

aws s3 sync dist/ s3://docs-production/ --delete \
  --exclude '*.html' --exclude '*.yaml' --exclude '*.json' \
  --cache-control 'public, max-age=31536000, immutable'

3. Force HTML to revalidate

HTML keeps its URL across deploys, so it must ask every time. max-age=0, must-revalidate does not mean “never cache” — the copy is stored and revalidated with an If-None-Match, so an unchanged page costs a 304 with no body rather than a full download:

aws s3 sync dist/ s3://docs-production/ --delete \
  --exclude '*' --include '*.html' \
  --cache-control 'public, max-age=0, must-revalidate'

4. Set no-store on previews

Preview hosts have the opposite requirement: correctness over speed, always. A reviewer must never see a cached copy of an earlier push:

aws s3 sync dist/ "s3://docs-previews/pr-${PR}/" --delete \
  --cache-control 'no-store'

5. Invalidate only what changed

With HTML revalidating, a purge is mostly belt-and-braces — but CDNs that serve stale content while revalidating in the background will show one stale response after a deploy. Purge the HTML paths, not the whole distribution:

aws cloudfront create-invalidation \
  --distribution-id "$CF_DISTRIBUTION_ID" \
  --paths '/*.html' '/index.html' '/openapi/api.yaml'

Purging /* invalidates the fingerprinted assets too, which is pure waste: their URLs changed, so nothing was cached under the new names anyway, and the old names are no longer requested.

The first request after a deploy With a long max-age on HTML the reader gets the old page; with must-revalidate the reader gets a conditional request and the new page. HTML with max-age=86400 reader CDN: still fresh never asks the origin old page served HTML with max-age=0, must-revalidate reader If-None-Match asks the origin ETag differs → 200 new page served unchanged pages still cost only a 304

Complete Working Example

A deploy script that applies all three policies in the right order and verifies the result:

#!/usr/bin/env bash
# deploy-docs.sh — publish with per-class cache policies, then prove it worked.
set -euo pipefail

: "${BUCKET:?set BUCKET}"
: "${CF_DISTRIBUTION_ID:?set CF_DISTRIBUTION_ID}"
: "${SITE_URL:?set SITE_URL}"

# 1. Fingerprinted assets first. Uploading these before the HTML that references
#    them means no reader can ever fetch a page whose assets are not yet present.
aws s3 sync dist/ "s3://${BUCKET}/" \
  --exclude '*.html' --exclude '*.yaml' --exclude '*.json' --exclude '*.xml' \
  --cache-control 'public, max-age=31536000, immutable'

# 2. The contract: stable URL, short life, revalidated.
aws s3 sync dist/ "s3://${BUCKET}/" \
  --exclude '*' --include '*.yaml' --include '*.json' \
  --cache-control 'public, max-age=300, must-revalidate'

# 3. HTML last, with --delete so removed pages disappear.
aws s3 sync dist/ "s3://${BUCKET}/" --delete \
  --exclude '*' --include '*.html' --include '*.xml' \
  --cache-control 'public, max-age=0, must-revalidate'

# 4. Purge only the stable-URL paths.
aws cloudfront create-invalidation --distribution-id "${CF_DISTRIBUTION_ID}" \
  --paths '/*.html' '/index.html' '/sitemap.xml' '/openapi/*' >/dev/null

# 5. Verify: HTML must not be long-cached, assets must be.
html_cc=$(curl -sI "${SITE_URL}/" | tr -d '\r' | awk -F': ' 'tolower($1)=="cache-control"{print $2}')
case "${html_cc}" in
  *max-age=0*|*no-cache*|*no-store*) echo "OK   html: ${html_cc}" ;;
  *) echo "FAIL html is long-cached: ${html_cc}" >&2; exit 1 ;;
esac

asset=$(curl -s "${SITE_URL}/" | grep -oE '/assets/[^"]+\.(js|css)' | head -1)
asset_cc=$(curl -sI "${SITE_URL}${asset}" | tr -d '\r' | awk -F': ' 'tolower($1)=="cache-control"{print $2}')
case "${asset_cc}" in
  *immutable*) echo "OK   asset: ${asset_cc}" ;;
  *) echo "FAIL asset is not immutable: ${asset_cc}" >&2; exit 1 ;;
esac

Expected output:

OK   html: public, max-age=0, must-revalidate
OK   asset: public, max-age=31536000, immutable

The upload order in that script is deliberate and is the part most often got wrong. Assets go up before the HTML that references them, so there is no window in which a reader can load a new page whose stylesheet has not been published yet. Deleting is confined to the HTML pass for the same reason — a --delete on the asset pass would remove the old fingerprinted files that pages still in someone’s cache are about to request.

Gotchas & Edge Cases

Cache-Control set at upload time is sticky. Object storage stores the header as metadata on the object, so re-uploading with a different policy only affects files that actually changed. A file whose content is identical is skipped by sync and keeps its old header forever. When you change a policy, force a metadata rewrite with --metadata-directive REPLACE rather than assuming the next deploy applies it.

Directory URLs and their index documents are two different objects. /guides/auth/ is served by rewriting to /guides/auth/index.html, and depending on the CDN the headers may come from the rewrite rule rather than from the object. Test the URL readers actually visit, not the file path.

The specification file is the most damaging thing to over-cache. Readers notice a stale page; nobody notices a stale openapi.yaml until an SDK is generated from a contract that is two releases old, or a mock server starts returning fields that no longer exist. Give it a short lifetime and an ETag, and treat it as the one asset worth purging explicitly on every deploy.

Compression and Vary interact with caching. A CDN that stores one variant per Vary header value can end up with several copies of the same page keyed on Accept-Encoding, which is fine, or on User-Agent, which is a cache-hit-rate disaster. Keep the Vary list as short as the content genuinely requires.

A service worker caches independently of everything above. If the site registers one, it has its own cache with its own rules, and a reader can be pinned to an old build no matter what the CDN says. Make the worker’s HTML strategy network-first, and version its cache name so a new deploy evicts the old entries.

When a reader reports stale content, work outwards from them rather than inwards from the origin. There are four caches in the path and only the outermost two are under your direct control:

Caches between origin and reader Origin storage, CDN, service worker and browser cache each hold a copy, and a stale page can come from any of them. origin bucket headers set here CDN edge purgeable service worker its own rules browser cache not purgeable you control the first two directly — the last two obey the headers you sent earlier

That asymmetry is the whole reason to get the headers right at upload time. A purge fixes the edge instantly, but a browser holding a copy with a long max-age cannot be reached at all, and the only remedy is to wait it out.

FAQ

Why do readers still see the old page after a deploy?

The HTML was cached with a long max-age. Because the HTML URL is stable, a cached copy has no way to know a new build exists, so it serves the old page until it expires. HTML must revalidate on every request; only fingerprinted assets may be cached long.

Do I need a CDN purge if HTML revalidates?

Usually not for correctness, since a revalidating request reaches the origin and picks up the new ETag. A purge still helps when the CDN is configured to serve stale content while revalidating in the background, and it is cheap insurance immediately after a deploy — just scope it to the stable URLs.

Should the OpenAPI file itself be cached?

Treat it like HTML — a stable URL whose content changes. Serve it with a short max-age and an ETag, because tooling fetches it repeatedly and a stale contract silently produces wrong SDKs and wrong mocks.