Static Site Generators in Production

Fixing Flaky Static Site Builds in GitHub Actions

A static site build should be deterministic: same commit, same output. In practice, many CI pipelines for static sites fail one run in ten or twenty for no apparent reason, and pass when re-run. The team learns to click "Re-run failed jobs", deploys slow down, and real failures get re-run too until someone notices they are not going away. Flakiness has causes, and on static sites they are few: network calls during the build, cache problems, resource limits on the runner, and tests that depend on timing.

This guide works through a 9% failure rate on the build workflow of an Eleventy documentation site with a CMS-backed changelog, Playwright tests and a link checker, and brings it below 0.5% by fixing each cause rather than retrying. It belongs to GitHub Actions for Automated SSG Builds.

Step 1: Collect the Failures

Clicking through failed runs gives anecdotes; the API gives a distribution. A short script pulls the last 500 runs of the workflow and, for failures, the failing step and the first error line from the job log:

gh run list --workflow build.yml --limit 500 --json databaseId,conclusion,headSha,createdAt > runs.json
jq -r '.[] | select(.conclusion=="failure") | .databaseId' runs.json | while read id; do
  gh run view "$id" --log-failed 2>/dev/null | grep -m1 -iE 'error|fail|timed out' | cut -c1-160 | sed "s/^/$id\t/"
done > failures.tsv
cut -f2 failures.tsv | sed -E 's/[0-9a-f]{7,}|[0-9]+ms|[0-9]+s/#/g' | sort | uniq -c | sort -rn | head

Of 500 runs, 46 failed. Excluding 9 genuine failures (broken code that also failed on re-run), 37 were flaky, and they grouped cleanly.

Number of flaky build failures by cause over 500 runs Horizontal bars. External link checker timeouts 13 failures. CMS API fetch failures during the build 9. Playwright tests timing out on animations or network idle 7. Out-of-memory kills of the build step 5. Cache restore producing a broken node_modules 3. Flaky failures by cause External link checker 13 CMS API fetch 9 Playwright timing 7 Out of memory 5 Broken cache restore 3
37 flaky failures out of 500 runs, grouped by failing step and first error line; 9 further failures were genuine and excluded.

The link checker validated every external link — about 2,100 — on every build. Some third-party sites were slow, rate-limited GitHub's IP ranges, or went down briefly, and each such event failed a pull request that had nothing to do with them. External link health is real information, but it does not belong in a pull request gate. The build now checks internal links only (deterministic, fast), and a scheduled weekly job checks external links and opens one issue with the broken ones, as in Crawling for Broken Links on a Schedule.

Cause 2: Fetching CMS Data During the Build

The changelog came from a headless CMS API at build time. Nine builds failed with timeouts or 502s from the API. Three changes made it robust:

async function fetchWithRetry(url, opts, tries = 4) {
  for (let i = 0; i < tries; i++) {
    try {
      const res = await fetch(url, { ...opts, signal: AbortSignal.timeout(10_000) });
      if (res.ok) return res;
      if (res.status < 500) throw new Error(`CMS ${res.status}`);   // don't retry client errors
    } catch (e) { if (i === tries - 1) throw e; }
    await new Promise((r) => setTimeout(r, 500 * 2 ** i));
  }
}

Bounded timeouts and exponential backoff absorbed most blips. For longer outages, the last successful response is cached with actions/cache keyed by date; if the CMS is still unreachable after retries on a pull request build, the build uses the cached copy and adds a warning annotation. Production builds from main still fail hard on a CMS outage, so stale content never deploys silently.

Cause 3: Playwright Tests Sensitive to Timing

Seven failures came from end-to-end tests waiting for networkidle (which an analytics beacon sometimes delayed) and from screenshots taken during a 300 ms menu animation. The fixes were the standard ones for Playwright: wait for specific elements or responses instead of networkidle, use web-first assertions (await expect(locator).toBeVisible()), and disable animations for tests with reducedMotion: 'reduce' plus a CSS override. Fixing Flaky Playwright Tests Against Preview Deploys covers these in detail.

Comparison of retrying jobs with fixing the causes of flaky builds Two panels. Retrying jobs hid the causes, doubled the time of failing runs, made genuine failures take two runs to report, and left the failure rate at 9 percent. Fixing causes moved external links to a schedule, added retries only around the CMS fetch, made tests wait on elements, raised memory headroom and fixed cache keys, bringing the failure rate below 0.5 percent. Retrying versus fixing Retry the job causes hidden failing runs take twice as long real failures re-run too 9% flaky Fix each cause external links on a schedule retry only the CMS fetch element-based waits memory and cache fixed <0.5% flaky
The only retry kept is a bounded one around the single network call that the build genuinely needs.

Cause 4: Running Out of Memory

Five runs were killed with exit code 137 during the build step — the runner's out-of-memory killer. Peak memory varied with what else ran on the job: the build ran in parallel with a Playwright server and Chromium processes from a previous step that had not exited. Separating the build into its own job, setting NODE_OPTIONS=--max-old-space-size=4096 explicitly so Node fails cleanly well below the runner's 16 GB, and reducing Eleventy's peak memory as in Fixing Eleventy Out-of-Memory Errors on Large Collections removed these.

Cause 5: A Cache That Restored the Wrong Thing

Three failures came from node_modules cached with a key based on package.json instead of the lockfile, so a cache from a slightly different dependency resolution was restored; native modules (sharp) then failed to load. Caching npm's download cache with actions/setup-node's built-in cache: npm (keyed on the lockfile) and always running npm ci fixed it — npm ci with a warm download cache took 14 seconds, barely slower than restoring node_modules. Caching node_modules in GitHub Actions compares the options.

Step 2: Watch the Rate

A weekly job recomputes the flaky rate from the API — failures that passed on re-run of the same commit — and posts it to the team channel. Anything above 1% gets a look. The number makes flakiness visible before people get used to clicking re-run again.

Weekly flaky build failure rate during and after the fixes Vertical bars of percentage of runs that failed and passed on re-run. Week 1 before fixes 9.1 percent. Week 2 after moving external links 5.2 percent. Week 3 after CMS retries 3.0. Week 4 after test fixes 1.4. Week 5 after memory and cache fixes 0.4. Week 8 0.3. Flaky failure rate by week 9.1% Week 1 5.2% Week 2 3.0% Week 3 1.4% Week 4 0.4% Week 5 0.3% Week 8
Computed from the GitHub API as failed runs whose commit later passed without changes.

Telling Flaky From Broken

The distinction matters for the numbers above and for the team's habits. A failure is flaky if the same commit later passes with no changes; it is genuine if it fails again. GitHub records re-runs as attempts on the same run, so the API distinguishes them directly: runs with run_attempt > 1 that concluded with success after a failed first attempt are flaky by definition. The weekly report counts those. It also lists the nine genuine failures from the original sample separately, because several of them had been re-run two or three times before anyone read the error — a cost of a team that has learned to expect flakiness. Once the flaky rate fell below 1%, developers stopped re-running by reflex and started reading failures, which is the real goal.

Measured Impact

MeasureBeforeAfterSource
Flaky failure rate9.1%0.3–0.5%GitHub API
Median build workflow time6 min 40 s4 min 10 sActions
Re-runs per week~14~1Actions
Time from merge to deploy, p9022 min6 minTimestamps

The workflow got faster too, mostly from removing the external link check (2 minutes 10 seconds) from every run.

Pitfalls & Rollback

  • Retrying whole jobs. It hides causes and slows every real failure.
  • External services in the pull request gate. Check them on a schedule.
  • networkidle and animations in tests. Wait for elements; disable motion.
  • Cache keys without the lockfile. Key on what determines the content.
  • Rollback: each fix is a workflow or script change; the weekly flaky-rate report shows whether reverting one brings failures back.

Conclusion

Flaky static site builds have a short list of causes, and counting failures by step and error message identifies them quickly. Moving external link checks to a schedule, retrying only the one required network fetch, making tests wait on elements, giving the build its own memory headroom and keying caches on lockfiles took this workflow from 9% flaky to under 0.5%, and made it faster.

FAQ

How do I find out why a GitHub Actions build fails only sometimes?

Collect the failures first. Export the last few hundred workflow runs with the GitHub API, group failed runs by the failing step and the first error line, and count. Intermittent failures almost always cluster into a handful of causes, and the step name plus error message identifies most of them without reproducing anything.

Why do builds fail when fetching remote data?

Because the build depends on a network service that is occasionally slow or unavailable: a CMS API, an image host, a package registry or a link checker's targets. Add timeouts and retries with backoff for required data, cache it so a failed fetch can fall back to the last good copy, and move checks of external services out of the build.

Can the Actions cache cause flaky builds?

Yes. A cache restored from a different dependency set, a cache that is partly written when two jobs save it at once, or a cache key that does not include everything that affects its content can all produce builds that fail on some runs. Include lockfiles and tool versions in keys and avoid saving the same key from parallel jobs.

Should I just add a retry to the whole job?

Only as a temporary measure. Retrying the job hides the cause and doubles the time of every failing run, and genuinely broken changes then take two runs to report. Retrying a specific network step with backoff is reasonable; retrying everything is not a fix.

Filed underFixesGitHub Actions