Static Site Generators in Production

Caching 404s and Redirects at the Edge

Static site caching guides focus on the responses that matter to readers: HTML and assets. The responses readers never want — 404s and redirects — often get no caching attention at all, and on a site with years of history they can be a surprising share of traffic. Vulnerability scanners probe for /wp-login.php and /.env thousands of times a day; old links from forums point at URLs from two migrations ago; bookmarks and search results still carry pre-migration paths that now redirect. If none of those responses is cached, every one of them travels to the origin.

This guide caches 404s and redirects at the CDN edge for a static site on CloudFront with an S3 origin and on Cloudflare, sets lifetimes that do not slow down fixes, and measures the change in origin load. It is part of CDN Caching Rules for SSGs. The site is a 2,300-page documentation site with 3,100 redirects from two migrations.

Prerequisites

Step 1: Measure What Reaches the Origin

A week of CloudFront logs grouped by status and edge result showed where origin requests came from:

ResponseShare of all requestsShare of origin requests
200 HTML and assets81%38%
40411%44%
301/3027%17%
other1%1%

404s were 11% of traffic but 44% of origin requests, because CloudFront's default error caching minimum TTL was set to 0 in this distribution and the S3 origin sent no cache headers on errors. Redirects, served by S3 website routing rules, were uncached for the same reason. Together they accounted for 61% of the origin load on a "fully cached" static site.

Response types as a share of all requests and of origin requests Paired bars. Successful responses were 81 percent of requests but 38 percent of origin requests. 404 responses were 11 percent of requests but 44 percent of origin requests. Redirects were 7 percent of requests but 17 percent of origin requests. Share of traffic versus share of origin load Share of all requests Share of origin requests 200 responses 81% 38% 404 responses 11% 44% Redirects 7% 17%
One week of CloudFront standard logs before any error or redirect caching.

Step 2: Cache 404s Briefly

On CloudFront, error caching is configured per distribution with custom error responses. Serve the site's 404 page with a 404 status and cache the error for five minutes:

"CustomErrorResponses": { "Quantity": 2, "Items": [
  { "ErrorCode": 404, "ResponsePagePath": "/404.html", "ResponseCode": "404", "ErrorCachingMinTTL": 300 },
  { "ErrorCode": 403, "ResponsePagePath": "/404.html", "ResponseCode": "404", "ErrorCachingMinTTL": 300 }
]}

The 403 mapping matters for S3 with origin access control: a missing object returns 403 from S3, which should appear to readers as a 404.

On Cloudflare, a Cache Rule can set status-code TTLs: cache 404 responses for 300 seconds at the edge, and do not let browsers cache them (browser TTL: respect origin, with the origin sending Cache-Control: no-store on 404s, or a Transform Rule setting it). Browsers should not remember 404s, because a reader who hits a page seconds before it is published should get it on refresh.

Five minutes was the balance point: long enough that a scanner requesting /wp-login.php every few seconds from many IPs hits the cache almost every time, short enough that a newly published page is never hidden for long — and deploys purge the cache anyway.

Step 3: Move Redirects to the Edge and Cache Them

S3 website routing rules redirect at the origin, so every redirect is an origin round trip, and S3 website endpoints do not support HTTPS or origin access control. The better design is redirects at the edge: a CloudFront Function with the redirect map, as in CloudFront Functions for Redirects, or Cloudflare Bulk Redirects. Edge redirects never reach the origin, so caching them is moot.

Where redirects must stay at the origin — an nginx server, a stub page, a Worker that computes them — add cache headers to the redirect response itself:

location = /docs/installation.html {
  add_header Cache-Control "public, max-age=3600, s-maxage=2592000" always;
  return 301 /guide/install/;
}

s-maxage=2592000 lets the CDN keep the redirect for 30 days; max-age=3600 keeps browsers from caching it permanently. Permanent redirects cached for a long time in browsers cannot be undone, which matters if a redirect turns out to be wrong.

Recommended edge and browser cache lifetimes for errors and redirects A matrix of response types. 404 responses cached 5 minutes at the edge and not at all in browsers. 403 from S3 mapped to 404 with the same lifetimes. Permanent migration redirects cached 30 days at the edge and 1 hour in browsers, or moved to an edge function. Temporary redirects not cached. 5xx errors cached 10 seconds at the edge at most. Cache lifetimes by response type Response Edge (s-maxage) Browser (max-age) 404 not found 300 s none 403 from S3 → 404 300 s none 301 migration redirect 30 days 1 hour 302 temporary redirect none none 5xx origin error 10 s at most none
Deploy purges clear all of these, so edge lifetimes can be longer than a writer's patience.

Step 4: Handle Scanners Before They Reach the Cache

A large share of the 404s came from a small set of paths probed by vulnerability scanners: /wp-admin/, /xmlrpc.php, /.env, /.git/config, /phpmyadmin/. For a static site these can be answered at the edge without even a cache lookup. On Cloudflare, a WAF custom rule blocks or challenges requests matching those patterns; on CloudFront, the CloudFront Function that handles redirects returns a 404 for them directly. That removed 72% of all 404 requests before caching was involved, and made the 404 reports described in Logging 404s at the Edge far more useful, because they now showed real broken links instead of scanner noise.

Handling of missing-URL requests at the edge A request arrives for a missing URL. If the path matches a known scanner pattern it gets a 404 from the edge function immediately. If it matches a redirect in the edge map it gets a 301. Otherwise the cache is checked, and a cached 404 is served. Only an uncached miss reaches the origin, which returns 404 and the edge caches it for five minutes. What happens to a request for a missing URL Missing URL request at the edge Scanner pattern? 404 from the function Redirect map? 301 at the edge Cached 404? served from cache Origin only true misses
After the change, 97% of 404 responses were answered without touching the origin.

Step 5: Keep Fixes Fast

Caching errors and redirects raises one worry: a fix — a new page at a URL that used to 404, a corrected redirect — must not be hidden by a cached response. Two practices keep that from happening. The deploy pipeline purges the cache for changed HTML and for redirect-map changes, as usual. And the lifetimes above are chosen so that even without a purge, a stale 404 disappears in five minutes and a stale redirect is corrected at the edge by a purge, with browsers holding it for at most an hour.

Soft 404s and Stub Pages

Some static sites implement redirects as stub HTML pages with a <meta http-equiv="refresh"> tag, generated by plugins such as jekyll-redirect-from or Hugo aliases. These return 200, so the CDN caches them like any page, but they cost a full HTML download plus a second navigation for every visitor, and search engines treat them as weaker signals than a real 301. Where the host supports edge redirects, generate a redirect map from the same data instead of stub pages and delete the stubs; on this site that removed 3,100 tiny HTML files from the build and made each old URL resolve in one hop. Also make sure the 404 page itself is lightweight: it is served to every scanner and broken link, and a 404 page that loads the full site search bundle multiplies that cost.

Measured Impact

MeasureNo error or redirect cachingAfter
Origin requests per day96,00031,000
404 requests reaching origin42,000/day1,300/day
Redirect requests reaching origin16,000/day0 (edge function)
Median time for a redirect, EU readers140 ms18 ms
S3 request and egress cost per month44 USD15 USD
Scanner requests reaching the cache layer~30,000/day~800/day

Pitfalls & Rollback

  • Long browser caching of 301s. Browsers keep them indefinitely; limit max-age or you cannot fix mistakes.
  • Caching 404s without purging on deploy. New pages may briefly 404 for readers who hit the URL early.
  • Returning 200 for the 404 page. A custom error page must keep the 404 status, or search engines index it.
  • Caching 5xx for long. A brief origin error cached for minutes becomes an outage.
  • Rollback: error caching TTLs and cache rules are distribution settings; setting them back to 0 or disabling the rule restores the previous behaviour immediately.

Conclusion

On a static site with history, errors and redirects can outnumber real page requests at the origin. Cache 404s briefly at the edge but not in browsers, move redirects into an edge function or give them long edge lifetimes, and answer scanner probes before they reach the cache. On this site that removed two thirds of origin requests and made redirects nearly instant.

FAQ

Should 404 responses be cached?

Yes, for a short time. Bots and broken links generate many requests for missing URLs, and without caching each one reaches the origin. A negative cache time of a few minutes removes almost all of that load while still letting a newly published page appear quickly.

How long should redirects be cached?

Permanent redirects from a migration can be cached at the edge for a long time, since they rarely change, but keep the browser cache lifetime moderate, because a 301 cached in a browser cannot be purged. Temporary redirects should have short or no caching.

Why are redirects reaching the origin at all on a static site?

When redirects are implemented at the origin, for example by an S3 website routing rule, an nginx rewrite or a stub HTML page with a meta refresh, the CDN forwards each request unless the redirect response itself carries cache headers.

Does caching 404s risk hiding newly published pages?

Only for the length of the negative cache time, and only if someone requested the URL before it existed. Purging the cache on deploy removes cached 404s along with everything else, so new pages appear immediately after a deploy that purges.

Filed underHow-tosAWS (S3 and CloudFront)