Fixing Missing Results in Pagefind Indexes

Pagefind is the default answer for search on static sites now: it runs after the build, indexes the HTML, and ships an index split into small fragments that the browser loads on demand. When it works, nobody thinks about it. When it does not, the failure is quiet — no error, no warning, just a reader searching for a page that exists and getting nothing. The site's own search becomes the place where content is hardest to find.

This guide works through missing results on a 2,600-page Hugo documentation site with a marketing section and a changelog, where support tickets reported that searches for changelog entries and half the API guides returned nothing. It shows how to measure what is in the index, the four causes found, and a CI check that catches regressions. It belongs to Search for Static Sites; the initial setup is in Indexing Hugo Sites with Pagefind.

Step 1: Measure What Is Indexed

Pagefind reports totals per language when run with verbose output:

npx pagefind --site public --verbose
# Indexed 1 language
# Indexed 1,412 pages
# Indexed 98,330 words

The site had 2,600 HTML pages. Comparing the indexed URLs with the sitemap gives the exact list of what is missing. Pagefind's Node API makes that a short script:

import * as pagefind from 'pagefind';
const { index } = await pagefind.createIndex();
const { page_count } = await index.addDirectory({ path: 'public' });
const files = await index.getFiles();            // includes the per-page fragments
const indexed = new Set(/* decode fragment URLs from files */);
const sitemap = await readSitemapUrls('public/sitemap.xml');
console.log(sitemap.filter((u) => !indexed.has(u)));

The missing 1,188 pages grouped by section immediately.

Cause 1: data-pagefind-body on Some Layouts Only

The docs layout marked its article element with data-pagefind-body to keep the sidebar and footer out of the index. The changelog, API and marketing layouts did not have it. Pagefind's rule is strict: once any page uses data-pagefind-body, pages without it are not indexed at all. Pagefind's verbose output does mention this, but the line was lost in a long CI log.

The fix was to add the attribute to the content element of every layout that should be searchable:

<main id="content">
  <article data-pagefind-body>
    <h1>{{ .Title }}</h1>
    {{ .Content }}
  </article>
</main>

That alone brought 1,082 pages back.

Cause 2: An Ignore Marker on a Wrapper

Even with the body marker, 42 blog posts had titles in the index but no body words. Their template wrapped the post body in a <div class="prose" data-pagefind-ignore> — added months earlier to exclude a "related posts" widget that had since been moved inside the same div. data-pagefind-ignore excludes the element and everything inside it. Moving the attribute onto the widget alone fixed it. A grep for data-pagefind-ignore across layouts, checking what each one wraps, is worth doing on any site that uses it.

Cause 3: Pagefind Ran Before Post-Processing Finished

106 changelog pages were indexed with their old content. The build ran hugo, then npx pagefind --site public, then a post-processing step that rewrote changelog pages to include data from the release API. The index described the pages as they were before the rewrite. Moving Pagefind to the very end of the build — after every step that writes HTML — fixed it:

{
  "scripts": {
    "build": "hugo --minify && node scripts/inject-releases.mjs && pagefind --site public"
  }
}

The same ordering problem appears with HTML minifiers, link rewriters and anything that runs in a host's post-processing; the index should be the last thing generated from the HTML.

Cause 4: Content Rendered Only in the Browser

The API guides each included an "Endpoints" table loaded by a script from a JSON file at runtime. Pagefind indexes the static HTML, so endpoint names never reached the index — the main reason readers could not find API pages by endpoint name. Rendering the table at build time from the same JSON with a Hugo data template put the content in the HTML. Where content genuinely must load at runtime, Pagefind's data-pagefind-meta or a custom record added through the Node API can include it, but rendering it statically is simpler and also helps readers without JavaScript.

Step 2: Weight What Matters

With everything indexed, a few ranking problems became visible: long changelog pages outranked short guides for common words. Two attributes helped: data-pagefind-weight="2" on guide headings and summaries, and a data-pagefind-filter="section:changelog" on changelog pages so the search UI could offer a "hide changelog" filter. Customizing the Pagefind UI shows the UI side.

Step 3: Check the Index in CI

The comparison script from Step 1 runs after every build and fails if more than 1% of sitemap URLs are missing from the index, or if any section's coverage drops below its previous level:

node scripts/check-search-coverage.mjs --min-coverage 0.99
# docs 1,180/1,180  api 410/410  changelog 640/640  blog 330/330  marketing 40/96 (allowed: data-pagefind-ignore-all)

Marketing landing pages are excluded deliberately with data-pagefind-ignore="all" on the body, so the check reads an allow-list of excluded sections. It takes 4 seconds.

Results That Exist but Rank Too Low

Some "missing" reports were not missing at all: the page was indexed but appeared on the second screen of results, and readers concluded it was absent. Two patterns caused most of these. Pages with very short bodies — a 90-word reference stub for a CLI flag — scored poorly against long pages that mentioned the same term many times. Adding the flag name to the page's <h1> and a one-sentence summary, both weighted by Pagefind's default heading boost, moved them to the top for their own names. And pages whose key term appeared only in code blocks scored low because code is indexed as ordinary text without any boost; a short prose sentence naming the term fixed those as well. Checking the top results for the 50 most common queries from the zero-result and low-click logs once a month now catches these before readers report them.

Measured Impact

Zero-result searches were measured with the privacy-preserving logging described in Tracking Zero-Result Searches.

Pitfalls & Rollback

• data-pagefind-body on some layouts. It silently excludes every page without it.
• Ignore markers on wrappers. They remove everything inside.
• Indexing before post-processing. Run Pagefind last.
• Runtime-rendered content. Render searchable content in HTML.
• Rollback: attributes and build order are independent changes; the coverage check makes any regression visible in the pull request.

Conclusion

Pagefind indexes exactly what the HTML says it should, which makes missing results a markup or build-order problem: a body marker on some layouts but not others, an ignore marker wrapping content, an index built before the HTML was final, or content rendered in the browser. Fixing those four on a 2,600-page site doubled the indexed pages, cut zero-result searches from 14% to 4%, and a 4-second CI check now keeps coverage complete.

FAQ

Why does Pagefind index some pages but not others?

The most common cause is data-pagefind-body. Once any page on the site uses it, Pagefind indexes only pages that have it, and silently skips every page without it. Other causes are running Pagefind before the generator has written all pages, pages excluded by data-pagefind-ignore on a wrapper, and pages whose html lang differs from the language being searched.

How do I see what Pagefind indexed?

Run Pagefind with the verbose flag, which reports the number of pages and words indexed per language and warns about pages without a body element. For detail, the Node API can index a directory and list each page's URL and word count, or you can query the generated index from a small script and compare against your sitemap.

Why are results found for titles but not for words in the body?

The body text may be outside the element marked with data-pagefind-body, inside an element marked with data-pagefind-ignore, or rendered by JavaScript after load. Pagefind indexes the static HTML, so content injected client-side is not indexed. Move the marker to the element that contains the article text and render content in HTML.

Do I need to rebuild the index after every deploy?

Yes. The index is generated from the built HTML, so it must run as the last step of every build, after the generator and any post-processing. A stale index from a previous build points at old content and misses new pages.

Related

• Parent: Search for Static Sites — choosing and running static search.
• Indexing Hugo Sites with Pagefind — the setup this site started from.
• Adding Pagefind to an Astro Site — the same tool in Astro.
• Customizing the Pagefind UI — filters and result display.
• Pagefind vs Lunr for Small Documentation Sites — when a single index is simpler.