Parallel Hugo Builds Across CI Jobs

Hugo is the fastest mainstream static site generator, and for almost every site one build on one machine is the right answer. A few sites are big enough that it is not: multilingual documentation with hundreds of thousands of pages, catalogue sites generated from product data, archives with decades of content. On those, a full build can run for 15 minutes or more on a standard CI runner and need more memory than the runner has. Hugo uses all available cores within a single build, so the remaining options are a bigger machine, or splitting the render across several machines.

Since version 0.124, Hugo supports segments: named subsets of the site defined in configuration, which a build can render on their own. This guide uses segments to build a 140,000-page, six-language product documentation site in four parallel GitHub Actions jobs, merges the output, and compares the result with a single large runner. It belongs to Hugo Build Times for Large Repositories; the configuration of segments themselves is covered in Splitting Large Hugo Sites with Segments.

Prerequisites

• Hugo 0.124 or later (this site used 0.139 extended).
• A CI system that supports job matrices and artifacts.
• A full build that already works, with its output file count recorded, to verify the merged output against.

Baseline

The 16-core runner was 2.6× faster on 4× the cores. Hugo's own phase timings (hugo --logLevel info) showed why: loading and assembling content took about 1 minute 40 seconds regardless of core count, and rendering scaled with cores.

Step 1: Define Segments That Partition the Site

Segments must together cover every page exactly once. Languages are the natural split for this site — six languages of roughly equal size, plus site-wide outputs:

[segments]
  [segments.en_de]
    [[segments.en_de.includes]]
      lang = '{en,de}'
  [segments.fr_es]
    [[segments.fr_es.includes]]
      lang = '{fr,es}'
  [segments.ja_zh]
    [[segments.ja_zh.includes]]
      lang = '{ja,zh}'
  [segments.global]
    [[segments.global.includes]]
      kind = '{home,sitemap,robotstxt,404}'
      lang = 'en'

English and German were paired because English is largest; Japanese and Chinese pages have smaller rendered output. The global segment holds the root home page, the sitemap index, robots.txt and the 404 page, which must be rendered exactly once.

A dry check confirms the partition before any CI work: building each segment locally into its own directory and counting files, then comparing the sum with the full build.

Step 2: Build Segments in a Matrix

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        segment: [en_de, fr_es, ja_zh, global]
    steps:
      - uses: actions/checkout@v4
      - uses: peaceiris/actions-hugo@v3
        with: { hugo-version: '0.139.0', extended: true }
      - uses: actions/cache@v4
        with: { path: resources/_gen, key: hugo-gen-${{ matrix.segment }}-${{ hashFiles('assets/**', 'content/**/*.jpg') }} }
      - run: hugo --minify --renderSegments ${{ matrix.segment }} -d out/${{ matrix.segment }}
      - uses: actions/upload-artifact@v4
        with: { name: seg-${{ matrix.segment }}, path: out/${{ matrix.segment }}, retention-days: 1 }

Each job loads the whole site — menus, cross-language links and .GetPage lookups need it — but renders only its segment. The image cache in resources/_gen is cached per segment, since each segment processes its own languages' images.

Step 3: Merge and Verify

A final job downloads every artifact into one directory and checks the result before deploying:

  merge:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with: { path: parts, pattern: seg-*, merge-multiple: false }
      - run: |
          mkdir site
          for d in parts/seg-*; do rsync -a "$d"/ site/; done
          count=$(find site -type f | wc -l)
          echo "files: $count"
          test "$count" -eq "$(cat .expected-file-count)"

rsync -a copies everything; fingerprinted assets (CSS, JS, shared images) exist in every segment with identical names and content, so later copies overwrite earlier ones harmlessly. The file-count check compares against a count committed to the repository and updated by a weekly full build, and catches the two real risks — a page in no segment (missing output) or a page in two segments (no error, but wasted work).

Step 4: Balance the Segments

The first split put English alone in one segment and grouped the other five languages; the English job took 7 minutes while the others finished in 3. Total time is the slowest job plus the merge, so balance matters more than the number of jobs. Rendered file counts per language guided the regrouping into the pairs above. A quick way to see per-language render cost is to build each language as its own segment once and time it.

Step 5: Handle Site-Wide Outputs

Three outputs needed care:

• Sitemaps. In a multilingual site, Hugo renders a sitemap per language plus a root sitemap index. Per-language sitemaps render in their language's segment; the index renders in global.
• RSS feeds. Per-language feeds follow their language; no change needed.
• Search index. The site's Pagefind index needs the complete HTML, so it runs in the merge job after rsync, not in a segment job. It adds about 50 seconds to the merge.

Measured Impact

Parallel segments were faster than the 16-core runner and cost about the same as the single 4-core build, because standard runners are billed at a quarter of the 16-core rate. The memory per job fell enough that builds no longer risked hitting the 16 GB limit.

Preview Deploys

Pull request previews use the same matrix, but most content pull requests touch one language. A small step at the start of the workflow inspects the changed paths and, when every change is under one language's content folder, builds only that language's segment plus global, then merges it over the latest production output downloaded from the artifact store. Preview deploys for single-language changes dropped to about 3 minutes. Changes to templates, configuration or shared data always trigger all segments, because they can affect every page.

When Not to Do This

Below roughly 20,000 pages or 3 minutes of build time, parallel segments add complexity for little gain: the repeated content loading phase in every job eats the savings, and the merge job adds a minute. Try Hugo partialCached for Faster Builds, the image cache and template profiling first. Segments shine when rendering, not loading, dominates — which on Hugo is true only for very large sites.

Pitfalls & Rollback

• Segments that overlap or leave gaps. Verify the merged file count on every deploy.
• Rendering site-wide files in several segments. Put home, sitemap index, robots and 404 in one segment.
• Unbalanced segments. Total time is the slowest job; balance by rendered output.
• Running the search indexer per segment. Index the merged output once.
• Rollback: a single hugo --minify job still works unchanged; the segment config is ignored when --renderSegments is not passed.

Conclusion

For a very large Hugo site, segments let you split rendering across parallel CI jobs while every job still sees the whole site. On 140,000 pages in six languages, four balanced segments and a verified merge cut deploy time from 18.5 minutes to just over 5, at the cost of the standard runner, and halved peak memory per job.

FAQ

Can Hugo build part of a site?

Yes. Hugo 0.124 introduced segments, defined in the site configuration with include and exclude filters on language, kind, path and output format. Running hugo with the renderSegments flag renders only the pages in the named segments, while still loading the whole site so cross-references and menus resolve correctly.

Is it faster to parallelise across jobs or use a bigger runner?

It depends on where time goes. Hugo already uses all cores within one build, so a bigger runner helps up to the point where loading content and memory dominate. Parallel jobs each load the whole site, so the load phase is repeated. On the 140,000-page site in this guide, four parallel jobs beat a single 16-core runner by about a third, because rendering dominated.

What breaks when merging segment outputs?

Anything rendered once for the whole site. The sitemap, RSS feeds, search index and the home page must be rendered in exactly one segment, and shared assets with fingerprinted names are identical across segments, so copying them is harmless. A file-count check after merging catches missing or duplicated output.

Do segments reduce memory per job?

Somewhat. Every job still loads all content, which is a large part of memory use, but rendering fewer pages reduces peak memory. On this site peak memory per job fell from 11 GB for the full build to 6.2 GB per segment.

Related

• Parent: Hugo Build Times for Large Repositories — the full set of Hugo build optimisations.
• Splitting Large Hugo Sites with Segments — segment configuration in depth.
• Matrix Builds for Multi-Site Monorepos — the matrix pattern for separate sites.
• Hugo Image Processing Cache in CI — caching resources/_gen per job.
• Multilingual Sites with Hugo — the language setup the segments split on.