CMS: instant and on-page search, its analytics, and a configurable page-contents nav - #446
Merged
Merged
Conversation
The widget could only search the whole site or a path prefix, and always navigated to a results page. It now carries two independent properties: - searchIn - the whole site, a section of it, or the page the widget sits on. "This page" resolves to the current request path, so a submit with no JavaScript still lands on a search of just that page. - instant - marks the form for progressive enhancement. The markup is unchanged either way; the marker attributes and the script only appear when it is on. Widgets already placed carry no searchIn, so an existing scope still reads as a section and an empty one as the whole site. No migration. Refs #426 AB#303114
GET /search/suggestions returns up to ten {label, url} rows for a term,
optionally scoped to a path. It is the typeahead source for the search
widget's instant option; the three existing suggestion endpoints are all
journey-domain and none of them search content.
SuggestAsync runs the same pass as SearchAsync, so scope handling, the
silent filters and URL canonicalisation cannot drift between what is
suggested and what a submitted search returns. It records no telemetry:
SearchAsync is the single place a search row is written, and a
per-keystroke writer would bury the analytics store and skew the volume,
latency and zero-result charts. A data-store failure yields an empty
menu rather than breaking the page it sits on.
Refs #426
AB#303114
instant-search.js upgrades a marked search form into a suggestion menu built on accessible-autocomplete, which the layout already loads. Two sources behind one menu: on a page search it indexes the page's own heading anchors and the copy beneath each one, and choosing a result moves both the URL and keyboard focus to that heading; otherwise it calls /search/suggestions and opens the page chosen. The form underneath is untouched, so a visitor without JavaScript, or one whose request fails, keeps the search box and button that were always there. The script does nothing when there is no content region, which is how the widget stays inert in the editor preview. Adds aria-controls to the generated combobox. The library still uses the ARIA 1.0 aria-owns spelling, which axe reports as a critical failure; the attribute survives the component's re-renders. Refs #426 AB#303114
Choosing a section with the arrow keys and enter left focus on the search box: the component returns focus to its own input as part of closing the menu, which undid the move. Deferring it to a later task lands after that, so arrow-and-enter now takes the reading position to the heading exactly as a click does. Caught driving the widget by keyboard rather than by click, so the E2E suite now exercises both paths. Refs #426 AB#303114
Every search widget rendered "search" as its input id, so two of them on one page produced duplicate ids and two labels pointing at the first input. Instant search made that worse: the listbox and status ids the autocomplete derives from the input collided too, which aims a screen reader at the wrong menu. Counted per request, so the first widget keeps the id it has always had. Refs #426 AB#303114
A typeahead has no server-side moment worth recording. The server sees a suggestion fetch per settled keystroke, and nothing at all for an on-page search, and neither is the event anyone wants: what matters is the query someone ended on, what they were shown, and whether they took any of it. Only the browser knows that, so it reports it. search_events gains surface (site, instant, instant-page), the host page an on-page search ran on, and the chosen key and position. A null selection is meaningful, not missing data: it means the menu was shown and none of it was any good. Sections get their own count rather than being filed as pages, and fold into the computed total so an on-page search offering three headings is not recorded as a zero-result search. Existing rows migrate to surface 'site', which is what they all were. POST /search/instant-analytics takes the browser's report. Every value in it is untrusted: the surface must be one this app declares, the query is sliced to the same 100 characters /search uses, the result list is capped, an unrecognised kind degrades to a page, and a malformed report is refused rather than written as a half-row that would read as a zero-result search. The session id comes from the server session, never the payload. The migration's generated Down was reordered: EF emitted the column drops before reverting the computed columns that reference them, which fails on the first statement. Refs #426 AB#303114
Nothing is sent while someone is still typing. One report goes out when the query is settled, which is any of: they chose something from the menu, they changed the query to something that is not a continuation of it, or they left the box, hid the tab or closed the page. Continuations supersede rather than accumulate, so typing e-v-i-d-e-n-c-e reports once, as "evidence". The middle trigger is the one worth having: someone shown three sections who takes none of them and types something else has said the results were no good, and a selection-only signal would throw that away. A report with no selection is that finding, not missing data. sendBeacon carries it, because it is the only transport that survives the page closing; it cannot set headers, so the antiforgery token travels in a form-encoded body. A failed send is swallowed - analytics must never break the page it is measuring. Refs #426 AB#303114
Two additions to the search-analytics surface. A surface filter on the shared window form, so every existing chart and table can be read as submitted searches, instant searches, single-page searches, or any combination. Unfiltered counts everything, which means the figures on the dashboard move the day this lands: they now include searches that were always happening but had nowhere to be counted. A single-page-search section listing the pages that carry a page-scoped widget, and what was asked of each one. It separates three outcomes a site search never distinguishes: nothing matched, something matched and was taken, and something matched and was left. The last is the number worth reading - the page was asked a question and had nothing useful to offer. Every read composes one surface-filtered source rather than naming search_events, and a test asserts that, because a query that named the table directly would report across every surface while the page around it claimed to be filtered. A long page path is one unbroken token and was setting the table's minimum width, pushing the numeric columns past the edge of the admin layout; it now wraps. Refs #426 AB#303114
The shared window filter is used by the single-page search tables, which draw no time series, so its bucket-size radios changed nothing there. A control that does nothing is worse than no control. Refs #426 AB#303114
"Instant search" and "Single-page search" read as two unrelated features, so someone who used the instant search on a page ticked "Instant search" and filtered their own searches out of the dashboard. They are now "Instant — the whole site or a section" and "Instant — this page only". The single-page section also warns when the surface filter excludes it, rather than reporting that nothing was searched for. An empty table because of a filter and an empty table because nobody searched are different facts and a reader could not tell them apart. Refs #426 AB#303114
The report for that case is sent with sendBeacon while the page unloads, so route interception is torn down before it can see it and the test has to read the row back from the dashboard instead. Verified by hand first against a real navigation: the row lands with the right host page and no selection. The single-page section test also asserted its page appeared in the paged list, which only held while that list was short - it addresses the terms drill-in by path now, and checks the list renders rows separately. Refs #426 AB#303114
The nav listed H2 and H3 only, so every H4 on a page was unreachable from it. H4 now nests under the most recent H3. Levels fall back rather than being dropped: an H4 with no H3 above it attaches to the current H2, and a heading with nothing above it at all becomes top-level. Authors skip levels — several of the guidance pages go straight from their title to H3s — and dropping those headings would leave sections the nav cannot reach, which is the job it exists to do. A new H2 resets the branch so a later section's H4 cannot attach to the previous section's last H3. The list markup recurses now instead of spelling out two levels, so the depth is the nav builder's decision rather than the view's. moj-side-navigation gives every nested list zero margin and padding, so until now the nesting existed only in the markup: an H3 sat at the same left edge, in the same size, as the H2 it belonged to. Each level is indented one step and set a size smaller. This changes how the contents nav looks on pages that already use it — previously flat lists will now read as the hierarchies they always were. AB#303114
The page-nav widget now carries a tick box per level, H1 to H6, with H2 and H3 ticked. The chosen levels form the hierarchy in the order they are chosen rather than by their absolute numbers, so a page that uses H2s and H4s but no H3s can tick just those two and get the H4s nested directly under the H2s. An unchosen level is ignored without breaking the chain around it. The nav is built from the page's own tree in the widget now, not read from the list the controller precomputes, because the levels are the widget's choice and two nav widgets on one page may legitimately want different ones. The precomputed list stays as the fallback for the editor preview, which renders a widget with no page around it. A widget placed before the tick boxes existed declares none of them, which means the default pair — not the empty list that unticking everything gives. Telling those apart is why the level reader checks whether the props are present rather than just reading each as a boolean. Each box carries a hidden companion posting "false". Without it an unticked box posts nothing, and the props builder keeps the registry default for a field it never sees, so unticking a level that defaults to on did not turn it off. Found by driving the editor form rather than posting props directly, and covered by a test that does the same. AB#303114
Three faults in the on-page search, all visible on one long page. The index only took headings that carried an id. The CMS anchors heading widgets, so a heading an author typed inside a rich-text block arrives as plain markup with none — 41 of the 77 headings on a page assembled from real guidance. Skipping them did not merely lose them: their text ran on into the previous anchored heading, so searching for a word underneath one offered a section some distance up the page. Searching "pregnant" returned "Life-limiting or critical illness". Every heading is indexed now, and given an id if it has none. Jumping to a section answered "where" but not "where exactly" — on a long section the word can still be paragraphs further down. Every occurrence is now wrapped in <mark>, the same element the search results page uses for its snippets, so the yellow means the same thing in both places. A new search clears the marks from the last one. The heading jumped to was outlined only when the browser's :focus-visible heuristic chose to, so the box came and went between jumps. It is asked for explicitly now, in GDS focus yellow. AB#303114
|
Review app for PR 446 was deleted |
The five test classes added here seed their pages under /help and build their URLs from it. Main has since moved runtime-created browser-test content to its own /development-testing root, so CmsSeedHelpers.HelpRootId no longer exists and this branch stopped compiling the moment the two met. Both halves move across: the parent the pages are created under, and the paths and widget scope built from it. Same reason as the pages that went before them — teardown is best-effort and the delete route is a soft delete, so anything that leaks stays in the tree and travels to the next environment inside a content export. Better that it leaks somewhere nobody has to ask what it is. FixtureContent gains the root's bare segment alongside its path, because a widget scope is written without a leading slash. AB#303114
The section-jump test read document.activeElement the moment the URL assertion was satisfied. Those two things do not happen together: the module sets the hash synchronously and then moves focus in a later task, deliberately, because the autocomplete puts focus back on its own input while closing the menu and the move has to land after that. So the URL matches first and the read catches whatever held focus at that instant. On a developer's machine the round trip to the browser is slow enough that the deferred task has already run; on the CI runner it is not, and the assertion came back with an empty string twice. It now uses the same retrying assertion the keyboard twin beside it already used. Confirmed by widening the deferral to 300ms: the old form then fails locally with exactly the CI message, the new one passes. AB#303114
ajs-uk-dev
approved these changes
Sep 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two strands of CMS work. The search widget learns to search as you type — including
searching just the page you are reading — and the analytics dashboard learns to count
those searches. The page-contents nav becomes configurable while it was being used to
test the first part.
Closes #426
AB#303114
Search widget
The widget could only search the whole site or a path prefix, and always sent the
visitor to a results page. It now has two independent properties.
Search in — the whole site, a section of it, or this page. "This page" resolves
to the page being viewed, so even with no JavaScript the button submits a search of just
that page.
Instant search — a tick box. Matches appear as the visitor types, with no button
press and no results page. On a page search the suggestions are the page's own sections
and choosing one moves to that heading; otherwise they are documents, and choosing one
opens it.
The two combine, which is where the request came from: instant search on this page gives
a find-as-you-type that jumps to the section, in the shape of GOV.UK finder-frontend's
search_with_autocomplete.How it works
The widget renders exactly the form it always did.
instant-search.jslayers a suggestionmenu over a marked form using
accessible-autocomplete, which the layout already loads. Avisitor without JavaScript, or one whose request fails, keeps the search box and button
that were always there.
A page search needs no server call — the headings are already in the DOM. A site or
section search calls the new
GET /search/suggestions, which runs the same pass as/searchso scope rules, the silent filters and URL canonicalisation cannot drift betweenwhat is suggested and what a submitted search returns.
Choosing a result marks every occurrence of the term on the page in
<mark>— the sameelement the results page uses for its snippets — and moves keyboard focus to the
destination heading, so the jump lands for a screen reader and not only for the viewport.
Accessibility
aria-controlsadded to the generated combobox. The library still uses the ARIA 1.0aria-ownsspelling, which axe reports as a critical failure.id="search"; the autocomplete extended that collision to the listbox and status ids,pointing a screen reader at the wrong menu.
:focus-visibleheuristic, which drew the box only sometimes.Search analytics
A typeahead has no server-side moment worth recording — the server sees a fetch per
settled keystroke, and nothing at all for an on-page search. What matters is the query
someone ended on, what they were shown for it, and whether they took any of it. Only the
browser knows that, so it reports it, once per settled query.
A report goes out when the query settles: they chose something, they changed the query to
something that is not a continuation of it, or they left the box, hid the tab or closed
the page. Continuations supersede rather than accumulate, so typing "evidence" reports
once. The middle trigger is the one worth having — someone shown three sections who takes
none of them and types something else has told us the results were no good, and a
selection-only signal would throw that away.
search_eventsgainssurface(site / instant / instant-page), the host page an on-pagesearch ran on, the chosen key and position, and a separate count for sections. Sections
fold into the computed total, so an on-page search offering three headings is not recorded
as a zero-result search. Existing rows migrate to
surface = 'site', which is what theyall were.
POST /search/instant-analyticstakes the browser's report. Every value in it isuntrusted: the surface must be one the app declares, the query is sliced to the same 100
characters
/searchuses, the result list is capped, an unrecognised kind degrades to apage, and a malformed report is refused rather than written as a half-row that would read
as a zero-result search. The session id comes from the server session, never the payload.
Dashboard
A surface filter on the shared window form, so every existing chart and table can be
read as submitted searches, instant searches, single-page searches, or any combination.
Every read composes one surface-filtered source rather than naming
search_events, and atest asserts that — a query naming the table directly would report across every surface
while the page around it claimed to be filtered.
A single-page search section listing the pages that carry a page-scoped widget and
what was asked of each. It separates three outcomes a site search never distinguishes:
nothing matched, something matched and was taken, and something matched and was left. The
last is the number worth reading — the page was asked a question and had nothing useful to
offer.
Page-contents nav
Picked up while using the nav to test the on-page search.
The nav listed H2 and H3 only. It now carries a tick box per level, H1 to H6, with H2 and
H3 ticked. The chosen levels form the hierarchy in the order they are chosen rather than
by their absolute numbers, so a page that uses H2s and H4s but no H3s can tick just those
two and get the H4s nested directly under the H2s. Levels fall back rather than being
dropped: a heading whose parent level has not appeared yet attaches to the nearest chosen
level above it. Authors skip levels, and dropping those headings would leave sections the
nav cannot reach, which is the job it exists to do.
The list markup recurses now instead of spelling out two levels, so depth is the nav
builder's decision rather than the view's.
This overlaps #425 (left navigation widget mis-navigates to h3 and h4 headings) without
necessarily closing it — that issue has no description, so I have not assumed it describes
this.
Notable fixes found while testing
default for a field it never sees — so unticking a nav level that defaults to on did not
turn it off. Each box now carries a hidden companion posting
false, placed after thelabel so the browser's first-value-wins ordering and the GDS adjacent-sibling styling both
still hold.
widgets, so a heading typed inside a rich-text block arrives as plain markup with none —
41 of the 77 headings on a page assembled from real guidance. Skipping them did not merely
lose them: their text ran on into the previous anchored heading, so searching for a word
underneath one offered a section some distance up the page. Every heading is indexed now,
and given an id if it has none.
input when closing the menu, so a click passed and arrow-and-enter did not.
Downdropped a column while the computed columns thatreference it still existed, which would have failed on its first statement.
Testing
The nine E2E failures are pre-existing and unrelated: they fail identically on unmodified
main, verified by building the web image frommainand re-running them. Seven areBackToTopTestshitting sample pages that exist with zero versions; two assert on searchhits for a term the local corpus does not contain. Both are now documented in #443.
New coverage spans every tier: widget registry and props, Razor source contracts for the
widget, the editor form and the nav partial, the suggestions endpoint's input guards, the
analytics mapper and the reporting endpoint, the surface filter and the single-page reads
against real Postgres, and browser tests for both jump paths, body-only matches, marking,
the no-results copy, section scoping, two widgets on one page, every nav level combination,
an editor round trip, a run with JavaScript disabled, and an axe scan.
The one gap worth naming: there is no JavaScript test framework in this repo, so
instant-search.jsis covered only by browser tests. The fetch-failure path and theout-of-order-response guard are not exercised.