Testing Search and Navigation Flows With Playwright

Most of a static documentation site cannot break at runtime: the HTML is what the build wrote. The parts that can break are the interactive ones — search, which depends on index files loading and a script parsing them; the mobile menu and its focus handling; the sidebar that expands to the current page; the version switcher that keeps readers on the same page in another version; and the 404 page that should help a lost reader recover. These are also the parts readers notice most when they fail, and the parts unit tests and HTML validation cannot see.

This guide builds a Playwright suite for those flows on a Starlight documentation site with Pagefind search, two product versions and a custom 404 page: 26 tests, run on every pull request in about 70 seconds. It belongs to Testing Static Sites in CI; the general setup is in End-to-End Tests for Static Sites with Playwright.

Prerequisites

• A built site served locally in CI (npx serve dist or npx astro preview).
• @playwright/test with Chromium, and WebKit for the mobile project.
• A short list of real search queries and the page each should find — from search analytics or support tickets.

Configuration

// playwright.config.ts
export default defineConfig({
  testDir: 'e2e',
  use: { baseURL: 'http://localhost:4321', trace: 'on-first-retry', reducedMotion: 'reduce' },
  webServer: { command: 'npx astro preview --port 4321', port: 4321, reuseExistingServer: !process.env.CI },
  projects: [
    { name: 'desktop', use: { ...devices['Desktop Chrome'] } },
    { name: 'mobile', use: { ...devices['iPhone 14'] }, testMatch: /nav|search/ },
  ],
  retries: process.env.CI ? 1 : 0,
  workers: 4,
});

reducedMotion: 'reduce' removes animation timing from tests, provided the site honours the preference.

Search: Known Queries, Keyboard and Empty State

Known queries live in a JSON file, so adding one is a one-line change:

import queries from './search-queries.json' assert { type: 'json' };   // [{ "q": "deploy to cloudflare", "expect": "/deploy/cloudflare/" }, …]

for (const { q, expect: path } of queries) {
  test(`search finds ${path} for "${q}"`, async ({ page }) => {
    await page.goto('/');
    await page.getByRole('button', { name: /search/i }).click();
    const box = page.getByRole('searchbox');
    await box.fill(q);
    const hits = page.locator('[role="dialog"] a[href]');
    await expect(hits.first()).toBeVisible({ timeout: 8000 });
    const hrefs = await hits.evaluateAll((as) => as.slice(0, 3).map((a) => new URL(a.href).pathname));
    expect(hrefs, `top results for "${q}"`).toContain(path);
  });
}

The assertion is "the expected page is in the top three", which survives small ranking changes while catching real failures. Two more tests cover keyboard use — open with Ctrl+K, arrow to the second result, press Enter, land on that page, and Escape returns focus to the button — and the empty state for a nonsense query. The keyboard test caught a regression where a theme update stopped the dialog returning focus, which the accessibility guide in Accessible Search UI for Static Sites describes as essential.

Navigation: Desktop Header and Mobile Drawer

On mobile, the menu is a drawer. The tests check it opens, traps focus, closes on Escape and on link click, and that links work:

test('mobile menu opens, traps focus and navigates', async ({ page }) => {
  await page.goto('/guides/');
  const btn = page.getByRole('button', { name: 'Menu' });
  await btn.click();
  await expect(btn).toHaveAttribute('aria-expanded', 'true');
  for (let i = 0; i < 30; i++) await page.keyboard.press('Tab');
  expect(await page.evaluate(() => !!document.activeElement?.closest('nav[aria-label="Main"]'))).toBe(true);
  await page.keyboard.press('Escape');
  await expect(btn).toBeFocused();
  await btn.click();
  await page.getByRole('link', { name: 'Reference' }).click();
  await expect(page).toHaveURL(/\/reference\/$/);
});

Pressing Tab thirty times and checking focus is still inside the menu is a simple, robust focus-trap test.

Sidebar State

The docs sidebar should expand the group containing the current page and mark the page with aria-current. Four tests visit pages at different depths and check exactly that, plus that the sidebar's scroll position brings the current item into view on long sections:

await page.goto('/reference/cli/deploy/');
const current = page.locator('nav[aria-label="Docs sidebar"] [aria-current="page"]');
await expect(current).toHaveText('deploy');
await expect(current).toBeInViewport();

Version Switcher

Readers switching versions should stay on the same page when it exists there, and land on the version's home page with a notice when it does not. Three tests cover both cases and the reverse direction. The switcher's logic is described in Versioning VitePress Documentation Without a Plugin; the same behaviour is tested here on Starlight.

404 Recovery

A missing URL must return a 404 status — tested by checking the response of page.goto — and the 404 page's search should be prefilled from the path, so a reader who mistyped /guides/instal/ finds "install" in one step:

const res = await page.goto('/guides/instal/');
expect(res!.status()).toBe(404);
await expect(page.getByRole('searchbox')).toHaveValue(/instal/);

Keeping the Tests Readable for Writers

The documentation team owns the search expectations file, not the developers. When a writer publishes a page that should be the answer to a common question, they add the query and the page's path to search-queries.json in the same pull request; if search does not rank it in the top three, the test fails and the writer adjusts the page's title or summary, which is usually the right fix anyway. Failure messages name the query and list the three pages that did come back, so a writer can act on them without reading test code.

The same principle applies to the navigation tests: they use role-based locators with the visible labels readers see ("Menu", "Reference", "On this page"), so when a label changes in the design, the test failure says exactly which label it expected. Renaming a navigation item is then a two-line change — the label and the test — reviewed together. Over four months, writers changed the expectations file 14 times and the navigation tests twice, without help from developers.

Measured Impact

The seven regressions: two search ranking changes that dropped a key page out of the top three, a focus-return bug in the search dialog, a mobile menu that stopped closing on link click after a theme update, a sidebar that lost aria-current on index pages, a version-switcher fallback that linked to a 404, and a 404 page served with status 200 after a host configuration change — the last one caught by the preview smoke subset.

Pitfalls & Rollback

• Asserting exact search result lists. Assert the expected page is near the top.
• Desktop-only navigation tests. Mobile menus break differently; test them separately.
• Animations in tests. Use reduced motion and web-first assertions.
• Running everything against the preview. Run the full suite locally, a subset on the preview.
• Rollback: tests are additive; a test can be skipped with test.fixme while its fix is pending, with an issue linked.

Conclusion

The interactive parts of a static docs site — search, menus, sidebar state, version switching and 404 recovery — are where runtime regressions happen, and none of them show up in build checks. A 26-test Playwright suite with known queries, keyboard and focus checks, a mobile project and a small preview smoke subset runs in about 70 seconds and caught seven regressions in four months that would otherwise have reached readers.

FAQ

What should end-to-end tests cover on a static site?

The parts that can break without the HTML changing: client-side search and its index files, menus and disclosure widgets driven by JavaScript, state such as the active sidebar item or a remembered theme, cross-page features such as version switchers, and recovery paths such as the 404 page. Static content itself is better checked by fast build-output tests.

How do I test search results without making tests brittle?

Assert on stable facts about known queries — a specific page appears among the first few results, a query with no matches shows the empty state — rather than on exact result counts or ordering of everything. Keep a short list of real queries with their expected top page, so tests double as a record of what search must get right.

Should navigation tests run on mobile viewports?

Yes. Mobile navigation usually differs in kind, with a menu button, a drawer and focus management, and is where most navigation regressions appear. Run the navigation suite in at least one mobile project, for example Playwright's Pixel 7 device, alongside desktop.

How long should these tests take?

Short enough to run on every pull request. The suite in this guide covers 26 flows in about 70 seconds with four workers against a locally served build. Tests against preview deploys take longer because of network latency, so the full suite runs locally in CI and a smaller smoke subset runs against the preview.

Related

• Parent: Testing Static Sites in CI — the testing pipeline.
• End-to-End Tests for Static Sites with Playwright — the base setup.
• Fixing Flaky Playwright Tests Against Preview Deploys — keeping the preview subset stable.
• Accessible Search UI for Static Sites — the behaviour the search tests protect.
• Synthetic Monitoring With Scheduled Playwright Checks — the same flows in production.